Image editing system for partially inserting video image into background

【0001】

[Technical field to which the invention belongs]

The present invention relates to image processing, particularly to a system for editing digital images.

【0002】

[Related technology]

For example, in the weather forecast of television broadcasting, it is possible to combine one video image and another video image to create a composite video image, like the background weather map and the weather forecaster in front of it. Often required. One of the well-known techniques for creating such composite video images is what is commonly referred to as "chroma key." This chroma key technology is so named because it uses the chroma, or color signal portion of the television signal, as a key to control the information in the composite image. A chroma key device is basically a video multiplexer for supplying a chroma signal from one of two video signals to a determination logic and selecting one input signal.

【0003】

In the above weather forecast application, the television camera is aimed at the weather forecaster in front of a vertical sheet with a given blue hue or shade, called a matte (matte). Hue is a unit of color, and is a perceptual scale that represents the range from red to yellow, green, blue, and then back to red. Images of the weather forecaster and Matt foreground source are supplied to the chroma key device. The chroma key device is programmed to select all video signals received from the camera, except for the image of the portion having a predetermined blue hue. Therefore, the chroma key device can effectively remove the forecaster's image from the matte image.

【0004】

At the same time as the above-mentioned operation is performed, an image of a weather map or an image of the earth seen from a satellite is given to the chroma key device as a background source. Images such as the name of the city and high temperature and low temperature are superimposed on the image of the earth as seen from this weather map or satellite. The chroma key device's decision logic selects the background source as the video output when a blue hue is detected in the foreground source, resulting in a composite image of the foreground and background so that the forecaster can see it. It is projected on the studio monitor. The forecaster will indicate the geographically corresponding location of the background source on the mat, and the viewer of the television program will see a composite image of the weather map and the forecaster. Such a composite image is the intended output of the chroma key device.

【0005】

However, as shown in US Pat. No. 4,811,084 by Belmares-Sarabia et al., A major drawback of chroma key systems is the occurrence of key errors. For example, if the weather forecaster is wearing a blue or striped suit, or even has blue eyes, the chroma key system may produce an inaccurately divided composite image. Furthermore, if the chroma key device is used in a state where the distance between the mat and the forecaster is too large and reflection occurs, a key error will occur, so it is necessary to restrict the movement of the forecaster.

【0006】

In order to solve this problem peculiar to chroma key, the patent by Belmares-Sarabia et al. Considers an image hue detection device whose key operation does not depend on a single color. For example, such devices distinguish between similar hues by limiting them by the hue bandwidth or saturation coupling that they can recognize.

【0007】

The device disclosed by Belmares-Sarabia et al. Uses analog signals to multiplex television signals. However, analog processing is not as flexible in image composition as digital processing. In digital processing, sophisticated image processing algorithms can be programmed to give a digitized image to modify or edit the image. Therefore, we provide a digital image system that can cut out a digital image of an object from the background and combine the digital object with different digital backgrounds or multiple backgrounds (composite) without suffering from the above problems and restrictions. To do is to contribute to the advancement of technology.

【0008】

Other ways to create composite images are Macintosh (Macintosh®) computers available from Apple Computer, Inc. and PC (PC) compatible computers available from IBM and other companies. There is a way to run image editing software. Examples of these programs are "Picture Publisher (registered trademark)" for PCs made by Micrografx, Inc. and Adobe Systems (registered trademark) made for Macintosh. There is "Adobe Photoshop (TM)" of Adobe Systems Incorporated). "Picture Publisher" is a registered trademark of Micrographics, and "Adove Photoshop" is a trademark of Adove Systems. "Macintosh" is a registered trademark of Apple Computer, Inc.

【0009】

With these programs, the user can apply the target image and cut out or move the target image in front of the background scene. However, these programs can only work on one object. These programs cannot form a multi-layer set of objects and backgrounds, and the user cannot move the object image to a different depth hierarchy of the composite image. In other words, you can put an image of a person in front of a house in the background scene, but not behind a fence at the same time. Therefore, providing a system capable of locating an object in front of or behind another object or background scene at a desired depth hierarchy of the composite image contributes to technological advances. It may also be desirable to make an object that has moved to a particular hierarchy transparent based on the desired attributes of that object. For example, one attribute of an object is hue, that is, the perceived hue. The leaves have a green hue, which can be set from opaque to transparent. Therefore, the sky, which was previously blocked by the leaves, can now be seen between the branches of the tree.

【0010】

When forming a composite image that includes a human face, there is a desire to replace the original human face with another person's face. Often, two faces are not the same size, for example, one face is a close-up and the other is not. Therefore, the creator of the composite image must reduce or increase the size of the replacement face. This work requires a lot of trial and error to get a satisfactory appearance. Therefore, it is desired to provide a simtem that can automatically determine the size of the face to be replaced and obtain a natural appearance without trial and error.

【0011】

Another aspect to consider when replacing one face with another is skin coloring. Suppose a user who creates a composite image replaces a dark-skinned face with a fair-skinned face. At that time, the remaining body parts, such as hands, arms, and legs, can be seen in the original image. Therefore, the composite image looks unnatural after the replacement. Therefore, there is a need for a system in which the skin tone of the replaced face automatically matches that of the original face without any manual intervention, and the composite image can provide a natural appearance.

【0012】

When replacing one face with another in the composition of a composite image, yet another aspect to consider is the alignment of the replacing faces. This alignment requires an iterative process to try different placements to get a natural and satisfying look. Therefore, there is a need for a system that can automatically align the face to be replaced in the optimum position so that the composite image has a satisfactory appearance.

【0013】

Occasionally, an image of an object is formed under one lighting condition and then superimposed on an image-formed background under another lighting condition. As a result, the composite image looks unnatural. Therefore, it would be more beneficial if the editing system could establish the same lighting conditions, or "gamma," throughout the composite image. For example, you may want to insert an image of an object formed under fluorescent light into a background formed under daytime light and maintain the same lighting conditions. The lighting condition in the composite image may be the third condition, for example, the moonlight.

【0014】

Therefore, there is a need to provide a digital image editing system that can separate the digital image of an object from the background in which the image of the object is formed. A further object of the present invention is to allow a digital image of a replacement object or multiple objects to be automatically sized, aligned, hierarchically aligned and placed in a given background at a desired depth. To provide a digital image editing system that can match the lighting conditions of the replacement object to one or more objects and backgrounds, and also to provide an easy-to-implement and cost-effective digital image editing system. To do.

【0015】

[Summary of Invention]

The present invention satisfies the above-mentioned needs and includes a system and a method for selectively combining digital images. The Simtem has a computer to which a device such as a video camera that generates a signal representing an image is connected. An object, such as a person, is placed in front of the video camera, which produces a signal representing the object and the background behind the object.

【0016】

Therefore, the signal from the camera has an object element representing an image of the object and a background element representing an image of the background. The object has an edge, and the computer detects this edge and separates the image outside the edge of the object (ie the background element) from the image inside the edge (ie the object element). To do.

【0017】

In one embodiment, the background has a single continuous hue, and the computer determines the position of the edges of the object based on the difference in hue between the object and the background. Based on this decision, the computer removes the background element from the signal.

【0018】

More specifically, a video camera creates a digital signal composed of a plurality of pixels arranged in a row by a process of digitization. Each pixel has a hue gamma, and each hue gamma has a numerical value that represents the lightness and darkness of that hue. The computer first determines the position of the edge of the object by checking the hue gamma value of the first pixel located at the edge of the top row of the video window (ie, around the video image). The computer determines the hue gamma value of the second adjacent pixel in the column and compares the difference with a predetermined difference.

【0019】

If the difference between the hue gamma of the first pixel and the hue gamma of the second pixel is less than the predetermined difference, the computer will align the hue gamma of the second pixel with the hue gamma of the first pixel and the second pixel. Compare the hue gamma of the third pixel, which is close to the pixel at 2. Similar comparisons are repeated. When the difference between two pixels exceeds a predetermined value, it indicates that a part of the edge of the object is present. The computer then repeats the process described above for the column immediately below the top column. If desired, the computer can also run the process as described above in different directions, such as moving up from the pixels in the bottom row.

【0020】

The computer continues this process, that is, the process of comparing pixels to pixels inside the border of the video window, until all the edges of the object are drawn. After the edges of the object have been drawn, the computer is ready to "strip" (ie remove) the background element from the object element by setting the background pixels to a given transparency value.

【0021】

In another embodiment, the background is a white substrate with a plurality of square boxes. Each box has multiple dots, and the processor records a map of its background in memory. When an object is placed in front of the background, the object shields the vertices of some boxes and some dots in the boxes. As a result, the processor can compare the stored map of the background with the digital video signal to determine the position of the object with the vertices and dots of the rectangular box shielded. When the position of the object is determined in this way, the processor removes the background image from the image of the object.

【0022】

After removing the background image from the object's image, the computer integrates the object's image into a preselected background image, if desired, that is different from the background in which the original object was imaged. .. More specifically, the computer can store one or more preselected background images in its memory. A computer combines multiple signals representing multiple layers of this stored background, additional text applied by the operator or user of the system, with an object element of the video signal to display a video display. The composite image is shown above. Therefore, if desired, the object image can be integrated into a preselected background image that is different from the background in which the original object was imaged, the user's text is added, and the composite image is displayed. can do. This image combination can be done by the user in real time, with the user watching the video display and using a mouse, keyboard or joystick or other input device connected to the editing system on a preselected background. In addition, the position and direction of the object can be adjusted.

【0023】

In other words, the computer basically functions as an image combiner by removing the image of the object from the digital video signal from the background image located behind the object. The processor then combines the image of the object with the preselected image and performs image combination in response to a signal formed in real time by the user. Specifically, a two-dimensional digital image representing a three-dimensional scene is stored in the memory of the processor. The processor then mixes the object into the selected background so that the object appears to be integrated into the 3D spectacle.

【0024】

Preferably, in order to mix the object with the preselected image, the processor averages the hue of the edge of the object with the hue of the portion of the preselected background adjacent to the edge. The processor then adjusts the hue of the object to be the same as the averaged hue.

【0025】

Further, the processor of the present invention adjusts the gamma for one or both of the object and the preselected storage background so that the object in the integrated image has the same lighting conditions as the preselected background. It can be expressed as imagined in. For example, the processor checks the gamma of a preselected background, adjusts the hue values of the pixels that make up the image of the object, and makes the object appear as if it were imaged under the lighting conditions of that background. To do so.

【0026】

The system according to the present invention can electrically connect a video printer for creating an image of an object to a processor. In addition, the system may have a money acceptor for receiving money in cooperation with the processor, and when money is inserted into this acceptor, the system may be activated in response. Therefore, as one application example, this system can be an electrical image booth provided in a public place so that it can be used by the general public.

【0027】

In addition to the systems described above, methods for video editing, i.e. integrating images of objects, such as people, into a two-dimensional display representing a given three-dimensional space are disclosed. According to the method of the present invention, a first digital image, including an object having a first hue and a background having a second hue, has an edge defined by the difference in hue between the object and the background. have. This edge is detected and the image outside this edge (ie, the background) is removed from the image. The object is then overlaid on a preselected background to form an integrated image of the object and a predetermined background.

【0028】

The system of the present invention automatically adjusts the dimensions of the object image so that the object image can be naturally integrated into the composite image. The computer compares the size of the original object on the background with the size of the object to be replaced and, if necessary, adjusts the size of the object to be replaced.

【0029】

The system of the present invention automatically aligns the object image so that the object image can be naturally integrated into the composite image. The computer uses a predetermined address of the original object on the background to transfer that address to a predetermined location of the replacement object.

【0030】

These objects and features of the present invention, together with the accompanying drawings (similar parts are similarly numbered), will be more fully clarified from the following description and accompanying claims.

【0031】

BEST MODE FOR CARRYING OUT THE INVENTION

First, according to FIG. 1, examples in the digital image editing system of the present invention are generally shown in 100. It is understood that the image system 10 can be used in virtually any application range in which an object can be separated from the background of a digital image and it is desirable to combine it with different backgrounds to form a composite image. One such application is shown in FIG. 1, in which the system 100 is mounted within the automatic electronic image system 104 by brackets 102. The device 104 includes a video camera 106 that is electrically connected to the system 100, such as the VKC-360 manufactured by Hitachi Corp.

【0032】

In FIG. 1, a video monitor or display 110, such as a Kodak 1310 RGB / CGA touch screen display, is mounted on the device 104. The monitor 110 is electrically connected to the system 100 to display a video image, such as the image of a person 112. As shown, the person 112 (user) using the system in FIG. 1 is sitting in front of a monochrome background 114. The monochrome background 114 may be in color.

【0033】

As shown in FIG. 1, device 104 also includes a toll receiver 116, such as the Rowe OB-A4 device. In this embodiment, the toll receiver 116 is used to receive cash as a payment. However, in other embodiments, other payment forms such as credit cards and tokens can also be received. In addition, device 104 includes printer 118 loaded with blank paper in card stock 120. When the correct amount is deposited in the toll receiver 116, or based on other activations, the image of the person 112 is printed on the sheet 120 and spit out to the printer exit 122. In this embodiment, the printer 118 is a Kodak SV6510 color printer.

【0034】

Now, according to FIG. 2, it can be seen that the system 100 includes various electronic devices. As shown in FIG. 2, the system 100 includes a digital computer 130. This is preferably an IBM PC compatible computer with an 80386 microprocessor operating at 33MHz and 8 megabytes (Mb) of memory. As shown, the computer 130 is electrically connected to the video camera 106 and the payment adapter 116 to receive the input signal. The computer 130 is also electrically connected to a suitable graphics video interface card 132, preferably Truevision's Targa (Targa®) + 16-32, which has 2 Mb of video memory. ing. Taga is a registered trademark of True Vision. The video memory on the graphics video interface card 132 stores digital representations of parts such as the person 112 (Fig. 1), the background, or user instruction screen information at various times. The graphics video interface card 132 is also electrically connected to the touch screen video monitor 110. Users of System 100 can respond to prompts given by the system by touching or then moving (drag) a location on the video monitor screen. An RS232 (serial format) digital connection from the video monitor 110 to the computer 130 allows user input to be moved to system 100.

【0035】

Computer 130 connects to printer 118 via a Centtronics compatible interface. The printer 118 is further connected to the video monitor 110 via an analog interface. FIG. 2 shows that an electronic storage device 134, such as a hard disk drive, can be connected to the computer 130 if desired. In this embodiment, the hard disk 134 has a capacity of 120 Mb.

【0036】

Here, with respect to FIGS. 1, 3a, 3b and 3c, the operation of the image system 100 of one preferred embodiment will be described. Figures 3a, b, and c represent the main or top-level control flow of System 100. For the particular embodiment shown in FIG. 1, execution begins at start step 202 in FIG. 3a, and a series of background scenes are displayed on the video monitor 110 (FIG. 1) to attract the attention of the user of System 100 (Attract). Mode) Proceed to step 204. The computer 130 periodically proceeds to decision step 206 to check if someone has touched the correct location on the monitor screen 110 (Figure 1) or has paid the payment adapter 116 (Figure 1). If there is no user input in step 206, computer 130 returns to step 204 and repeats the "attract" mode.

【0037】

If there is user input in step 206, determination step 208 checks if the user has charged the payment adapter 116, in which case computer 130 proceeds to determination step 210. Judgment step 210 tests whether the fee is reasonable and genuine. In this embodiment, System 100 receives only $ 5 bills. However, by using a computer 130 configuration file (not shown), the technician can change the payment type to, for example, five dollar bills. In step 210, if the payment is rejected for any reason, the computer returns to step 204 again.

【0038】

In step 212, the audio file and audio drive from the sound board of computer 130 are activated, instructing the user to insert a $ 5 bill. For the sound board, Creative Labs, There are Sound Blaster (registered trademark) of Inc. and Thunder Board (TM) of Media Vision, Incorporated. Sound Blaster is a registered trademark of Creative Loves, Inc. Thunderboard is a trademark of Media Vision. A video clip with payment instructions is stored on storage disk 134 (FIG. 2) and displayed on the monitor screen 110 for 15 seconds. A check is made in step 214 to determine if payment was made during that 15 seconds. If not paid, the computer returns to step 204 again. If the fee was paid in step 214, test whether the fee is correct in step 210. If the charges are correct, the computer proceeds to step 216, where RS232 approval for computer 130 is sent. The main control flow leaves FIG. 3a through connector A218 and continues to step 220 in FIG. 3b.

【0039】

In step 220, a live video with the digital mask 140 (FIG. 4a) is projected on the video display 110. An audio file from computer 130 activates user instructions. FIG. 4a shows what the user 112 sees on the video display 110. Box 142 outlines the digital mask 140 and defines the specific range in which the user 112's head should be. If the head is not within the range of the box 142, the user is voiced to instruct the user 112 to sit deeper or to raise or lower the given chair (not shown). If the user 112's head fits within the box 142, there is no need to further reduce the size of the head. The digital mask 140 is a monochrome hue formed by a technician.

【0040】

Moving from step 220 to step 222 (FIG. 3b), computer 130 activates video camera 106 (FIG. 2). It then captures a video signal representing an object, such as a person or user 112, and a background (FIG. 4a) in which the object, such as a continuous hue background 114, is imaged. Computer 130 proceeds to determination step 224 to determine if this is the first captured image or image. If so, computer 130 returns to step 220 and step 222 to take the second pose of user 112. If step 224 determines that two images have been captured, computer 130 proceeds to step 226, where both video images are digitized. Of course, the system can be modified to receive and process just one or more images.

【0041】

As shown in FIG. 4a, only a portion 142 of the camera view is digitized for processing by the computer 130. Normally, the signals from camera 106 are analog signals, which are digitized by a video frame grabber on TrueVision's Taga + card 132 into pixels or a two-dimensional matrix of image 144 (shown in Figure 4b). In this embodiment, the pixel 144 matrix is 512 x 486, which is the digital video square standard, but other matrix sizes of 640 x 480 or larger may be used.

【0042】

After both images have been digitized, the computer moves to step 228. There, both images are displayed on the video display 110 (Figure 1), the audio file is selected for the user 112, one of the two for the rest of the process, and in step 230 to that image on the monitor screen 110. Instruct to touch. The selected image 144 is stored in the video memory of the graphic interface card 132 (FIG. 2). After user 112 makes the necessary selections in step 230, computer 130 proceeds to step 232. In step 232, a different background scene is displayed on the video display 110. For example, in the background scene, members of a sports team wear sports equipment individually or in groups in various poses. In this embodiment, the computer 130 displays some preselected background scenes, but in other embodiments, the user may choose another sports team or the like. In other embodiments, the user can also choose which person from a group of people in a scene wants to replace his photo. This selection is made by pointing to a person, specifying a name, the person's job title, etc., or choosing a place replacement. User 112 is prompted in step 232 to select the desired background scene by touching the appropriate background on the video screen 110 (FIG. 1).

【0043】

The computer performs function 234 to strip the portion of the image selected by the user in step 230 while the user decides which background scene to select. Function 234 will be described in detail later. When the user 112 touches in step 236 to select an appropriate background on the monitor screen 110 (FIG. 1), the computer 130 receives an interrupt telling it that it has been selected. In the meantime, when feature 234 returns from execution, computer 130 proceeds to step 238, where the result of feature 234, such as the image of the user's head or neck along the stripped background pixels, is the graphic interface card 132 (Figure). It is written to the video memory of 2). If the computer 130 receives an interrupt suggesting the selection of the background scene from step 236, the control flow 200 proceeds to step 240.

【0044】

In step 240, a personal computer keyboard and a text entry screen following instructions to user 112 are displayed on the video monitor 110. An audio file from computer 130 is activated to give the user verbal instructions. The user is prompted to include text such as the user's name or other name in order to personalize the final composite image. While considering the text to be entered by the user 112, the computer 130 proceeds to step 242 and sends the background scene selected by the user in step 236 to the frame storage of the memory of the printer 118. As shown in FIG. 4e, in this embodiment, in the background scene sent to the printer 118, the head and neck of the preselected person 145 have disappeared from the scene stored in the original computer. There will be. In another embodiment, the person in the scene stored on the original computer selected to replace the user (among a plurality of people) is stored in the printer frame without a head or neck. Sent to the department. While the background is being written to the printer's frame storage in step 242, the user can press a selection on the touch screen monitor 110 in step 244 to select the character that creates the desired text. The touch screen monitor 110 is given a place to display the end of text, which causes the computer to proceed to decision step 246. In step 246, check if valid text has been entered. If not filled in, computer 130 returns to step 242 and is given a selection on the touch screen monitor 110 to correct the text.

【0045】

If determined to be valid in step 246, computer 130 proceeds to step 248, where the facial image selected by user 112 in step 230 (Figure 4b) is the original monochrome when the user's photo was taken. It is displayed on the video monitor 110 along with the background. After the end of step 248, the main control flow 200 leaves FIG. 3b through connector B250 and continues to step 252 in FIG. 3c.

【0046】

In step 252, the computer 130 draws a parallel reference line on the video display 110 and prompts the user 112 to touch and drag (touch and move) the reference line through the pupils of both eyes. .. At step 254, user 112 positions the parallel line on the pupil and presses a button on the touch screen monitor 110 to signal the end of this step. Moving from step 254 to step 256, the computer 130 draws a reference cloth on the video display 110 and prompts the user 112 to touch the reference cloth and drag it under the chin and to the center of the neck. At step 258, user 112 positions a reference cloth in the center of the neck and at the bottom of the chin, and presses a button on the touch screen monitor 110 to signal the end of this step 258. The monitor 110 has a display similar to that in FIG. 4b, but includes a monochrome background 114.

【0047】

Upon completion of step 258 in FIG. 3c, the computer calls function 260 to match the size of the user's face as determined by steps 252 to 258 so that it can be replaced with the preselected person 145 (from step 236). I take the. Function 260 will be described after the rest of the main control flow has been described. After function 260 returns, the computer calls function 262 to change various gamma values.

【0048】

The overall gamma has a number of attributes. For example, Hue (H), Saturation (Saturation) (S), Brightness (L), Intensity (I), Contrast (C), Red (R), Green (G), Blue (B), and HSL. , HSI, HSC, RGB, etc., and combinations thereof. The top-level gamma feature 262 can change any combination of gamma attributes for each pixel, image area or overall image. For example, the hue, saturation, and intensity of the image area can be changed. Other actions performed by the top-level gamma function include stripping, fuzzing and blending, transparency / opacity and pixel enhancement. These functions, and the devices and methods for achieving them, will be described later in this specification. For example, the user may choose to only emphasize pixels of a particular blue hue and blend pixels of a certain saturation level. Any combination of gamma attributes and actions is possible.

【0049】

As with the other elements of the invention, in order to better understand the application of top-level gamma function 262, it should be recognized that the final composite image is processed by computer 130 layer by layer. .. Note that each pixel on the layer has X and Y Cartesian coordinates. Thirty-two layers are used in this example, but in other examples it is limited by the amount of memory, but there may be more layers. The final composite image can be seen with the layers stacked on top of each other. The number of layers provides the Z coordinate, and the original background scene has a Z coordinate of zero. The objects in the original background scene are chosen to have a higher or similar priority, and thus be given a higher number of layers and Z coordinates. Other objects, such as the image of user 112, are assigned a Z coordinate and placed in front of or behind an object from a background scene pre-moved from hierarchy zero (depending on that Z coordinate).

【0050】

For example, consider the final composite image with four layers (Z = 0 to 3), as shown in Figure 4h. If pixel 150 at a particular X, Y Cartesian coordinate address in the top hierarchy (Z coordinate = 3) has a transparency attribute, then the pixel below it (Z = 2) at the X, Y coordinate. 151 can be seen if the attributes are opaque. However, if pixel 151 with Z = 2 is similarly transparent, then pixel 152 at the X and Y coordinates of Z = 1 can be seen if its attributes are opaque. In order for the background and pixel 153 on the Z = 0 hierarchy to be visible, all pixels on the higher hierarchy that take its X, Y Cartesian coordinate address must have the transparency attribute.

【0051】

Furthermore, in FIG. 4h, some objects have been moved from the original background hierarchy (Z = 0). Two kneeling players 154 have moved to the top level (Z = 3) and hat 155 has moved to the Z = 1 level. The head of user 112 is placed as object 156 in the hierarchy of Z = 2. In the background scene, the head part 158 of the person 157 whose head should be replaced is transparent. The final hierarchical composite image 159 shows the user's head 156 wearing a hat 155 on the body of person 157.

【0052】

The gamma attributes and actions described above can be performed layer by layer, layer by layer. For example, a user strips pixels of a particular red hue from layers 1, 2 and 3, then sharpens a pixel with a certain intensity value at layers 0 and 1, and then at layers 1 and 3 all pixels. Can be blurred (fuzzed) to the desired saturation level. Top-level gamma function 262 is detailed below.

【0053】

After returning from feature 262, computer 130 calls feature 264 to match the skin color of user 112 with that of selected background person 145 (Figure 4e). Function 262 is described below.

【0054】

After feature 264 returns, computer 130 calls feature 266 to match the attributes of the user image with that of the background scene (Figure 4e) as selected in step 236. Function 266 is described below.

【0055】

In this embodiment, after function 260 returns, the main control flow 200 proceeds to function 268 and bypasses functions 262, 264, 266 (this bypass is not shown in FIG. 3c). However, it should be recognized that features 262, 264, 266 are included in other embodiments.

【0056】

After function 266 returns, computer 130 calls function 268 for pixel enhancement. Function 268 is described below.

【0057】

After the function 268 returns, the computer 130 calls the function 270 and positions the object in the selected background scene (Fig. 4e). Function 270 is described below.

【0058】

After returning from feature 270, computer 130 displays a deversion screen, such as the team logo, on video monitor 110 in step 272 with a message that the final image is ready soon. Computer 130 proceeds to step 274 while the deversion screen is displayed. In step 274, computer 130 calls the layering function to layer the image of the user's head or neck on the background scene selected in step 236. Function 274 will be described later.

【0059】

After the function 274 returns, in step 276 the computer 130 sends the user's head and neck blocks processed by the stacking function to the frame buffer of the printer 118 in overlay mode. In overlay mode, the block currently sent to the framebuffer starts at the position defined for the current block and overwrites the previously existing information in the buffer. The information that previously existed in the framebuffer outside the area of the overlaid blocks remains unchanged. The final composite image is stacked in printer memory. In advance, the background scene is sent to the printer 118 in step 242.

【0060】

After completing step 276, computer 130 proceeds to step 278, where the nearly completed composite image is sent from the printer memory to the video display 110. The computer 130 then advances to recall the stacking function 274 for overlaying the personalized text that the user filled out in step 244. When the stacking function 274 returns, computer 130 proceeds to step 282 where the personalized text is sent to the framebuffer of printer 118 in overlay mode. At this point, the final composite ends in printer memory. Computer 130 proceeds to step 284, where personalized text is sent from printer memory to the video display 110, displaying the final composite image to user 112 for a few seconds. While the composite image is displayed, computer 130 proceeds to step 286, in which an audio file from computer 130 behaves to thank user 112 for using system 104. Then, in step 288, the computer 130 prints the final composite image on the cardstock 120 and signals the printer 118 to output the printed matter from the printer exit 122. After the final composite image is printed in step 290, the main control flow 200 returns to step 202 and restarts the entire process.

【0061】

Now, in FIG. 5, another embodiment of the digital image editing system of the present invention is outlined in 1010.

【0062】

FIG. 5 shows an application showing that the system 1010 is mounted within the automatic postcard system 1012 by brackets 1014. Some of the following features of System 1010 have also been described for System 100 in Figure 1. The device 1012 includes a video camera 1016 such as Hitachi's VKC-360, which is movably mounted on the device 1012 and electrically connected to the system 1010. The device 1012 also includes a handle 1018 connected to the camera 1016 for the purpose of manually operating the camera on the device.

【0063】

In Figure 5, a video monitor 1020, such as a standard RGB / CGA display, is mounted on device 1012, which is electrically connected to system 1010 to display a video image, such as that of model 1022. Has been done. As illustrated, model 1022 in FIG. 5 stands in front of a monochrome background 1024. The monochrome background 1024 can be any color you want.

【0064】

As shown in FIG. 5, device 1012 includes a toll receiver 1026, such as the Rowe OB-A4 device. The device 1012 also includes a printer 1028 containing a blank sheet of postcard stock 1030. When the correct amount is deposited in the toll receiver 1026, the image of model 1022 is printed on sheet 130 and spit out. In this embodiment, the printer 1028 is a Kodak SV6510 color printer.

【0065】

In addition, one or more controls 1032 are attached to device 1012 and connected to system 1010. Control 1032 in FIG. 5 is a button that allows model 1022 to manually position the image of model 1022 on one image of a plurality of preselected backgrounds electronically stored in system 1010.

【0066】

Figure 6 shows the system 1010 used for non-monochrome backgrounds. More specifically, the system 1010 is used with a background 1034 consisting of a monochrome (eg white) substrate 1036. A plurality of black boxes 1038 are printed on the substrate 1036, and inside each square, a plurality of black dots 1040 are printed on the substrate 1036 in a predetermined pattern. As shown, box 1036 is provided with a plurality of vertices 1042. In one embodiment, each box 1036 has a side of about 2 inches and each dot 1040 has a diameter of about 0.25 inches. Therefore, the background 1034 has a predetermined pattern, which is electronically stored in the system 1010. This background 1034 is also pre-stored in system 1010.

【0067】

With reference to Figures 6, 7 and 8, the behavior of the system 1010 when the checkerboard background 1034 is used is recognized. In Figure 7, function 1085 is initiated and proceeds to block 1086. The operation of computer 130 on blocks 1086 and 1088 in FIG. 7 is essentially as described above. See page 14, lines 19-31.

【0068】

From block 1088, computer 130 proceeds to block 1090. At block 1090, computer 130 starts at the upper left and lower right corners of the pixels of the digitized video image at the same time, performing a "coarse grain background strip". In particular, computer 130 compares the vertices 1042 of the memorized image 1054 with the memorized map of background 1034 represented in FIG. 6 and in FIG. 8 with Roman numeral I. This memorized background map is a map in X, Y Cartesian coordinates of dots 1040 in box 1038 and background 1034. In other words, the computer 130 is a block 1090 that determines which part of the image 1054 contains the box 1038 and stores that part as the background.

【0069】

From block 1090, computer 130 proceeds to block 1092, where computer 130 removes the portion of image 1054 that matches the background map stored in memory based on the comparison at block 1090. Especially at block 1092, computer 130 removes the portion of image 1054 that forms the edge of the complete (four-sided) box 1038. Computer 130 also removes the portion of image 1054 within the complete box 1038 (ie, such as the white substrate 1036 and the dot 1040 inside the image box 1038).

【0070】

The specific coarse-grained background strip operation of computer 130 at block 1090 is well illustrated in Figure 9a. Computer 130 proceeds to block 1093 to start the coarse-grained background strip process. Specifically, computer 130 determines if the current pixel, the test pixel, should be the edge of box 1038, as shown in determination block 1094 in FIG. 9a. As mentioned earlier, the computer 130 first selects the upper left and lower right pixels as test pixels. After performing the following coarse grain strip test, for example, on the upper left pixel, the computer 130 uses the coarse grain strip feature described below to detect the next pixel in the top row if no edges are detected, or if edges are detected. Select the leftmost pixel in the next lowest column. If you proceed from the lower right pixel, one of ordinary skill in the art will find that the sequence of test pixel selection is a mirror image of that of the upper left pixel sequence. More specifically, after testing the bottom right pixel, the computer 130 will determine the next pixel in the bottom row if no edges are detected, or the rightmost pixel in the next highest column if edges are detected. select.

【0071】

At block 1094, computer 130 compares the test pixel with the portion of the stored background map associated with the test pixel in X, Y Cartesian coordinates. If this comparison suggests that the test pixel should be the same color as the edge of the box (that is, the test pixel should be part of the box), computer 130 goes to decision block 1096.

【0072】

At block 1096, computer 130 determines if the current pixel of image 1054 is the same color as the edge of box 1038, i.e. black. If so, computer 130 goes to block 1098 and stores the current test pixel as a background pixel. From block 1098, computer 130 proceeds to block 1100, where it selects the next pixel in the current test pixel column.

【0073】

If computer 130 determines that the test pixel is not black in block 1096, computer 130 proceeds to block 1102. Computer 130 then maps the test pixels as edge pixels, making sure that the test pixels are part of edge 1066 of object 1022 in the digitized image 1054. From block 1102, computer 130 proceeds to block 1100 and the next pixel is selected as above.

【0074】

In block 1094 of FIG. 9a, if computer 130 determines that the current test pixel is not part of box 1038, computer 130 proceeds to decision block 1104. In decision block 1104, computer 130 determines if the test pixel is white, that is, if the test pixel is part of the white portion 1036 of the background 1034. If not white, computer 130 proceeds to block 1102 and performs the above operation. Otherwise, computer 130 goes from block 1104 to block 1098.

【0075】

At this stage of the process, test pixels are recorded either as background 1034 pixels in block 1098 or as object pixels 1022 in block 1102. Computer 130 proceeds from block 1100 to decision block 1106. Computer 130 then determines if the test pixel is later than the last pixel in the column. If so, computer 130 proceeds to block 1108. There, the computer 130 is the first pixel in the column immediately following the tested column (that is, the leftmost pixel for processes that started in the upper left corner of the digitized image, and also the lower right pixel. Select the rightmost pixel for processes that started in the corner). Otherwise, computer 130 regresses to decision block 1094 to process the new test pixel.

【0076】

Computer 130 proceeds from block 1108 to block 1100. Computer 130 then determines if the column just tested was the last column. In other words, computer 130 is in decision block 1110, if a particular process is part of a process that started in the upper left corner of the digitized image, then the just-tested column is the digitized image. Determine if it is the lowest column of. On the other hand, if a particular process is part of a process that started in the lower right corner of the digitized image, then the computer 130 asks if the column just tested is the top row of the digitized image. Judge whether or not.

【0077】

If the current pixel is not in the last column, computer 130 regresses to decision block 1094 to process the new next test pixel. On the other hand, if it determines that the computer 130 has tested the last pixel in the top row, the computer 130 exits the coarse-grained background strip function of FIG. 9a with function 1111 and returns to block 1112 of FIG.

【0078】

Again, with respect to FIG. 7, in contrast to the coarse-grained background strip subroutine described above, the process performed by computer 130 in blocks 1112 and 1114 can be thought of as the fine-grained background strip function. The image remaining after computer 130 performs the coarse-grained background strips defined in blocks 1090 and 1092 is represented by Roman numeral III in FIG. There, only a small part of the background 1034 (that is, not within the image of the complete box 1038) remains around the image of model 1022. The small portion remaining in background 1034 is removed by the process running in blocks 1112 and 1114. There, the dots in the remaining background image are compared to the background map and removed from the digital image by computer 130.

【0079】

The fine-grained background strip function of block 1112 is better understood in Figure 9b. Computer 130 starts at function 1115 and proceeds to decision block 1116. Starting with decision block 1116 in Figure 9b, computer 130 determines if the current pixel should be a dot. In selecting the current pixels, it is understood that the computer 130 performs a selection routine similar to the routine described in the case of coarse-grained background strips. Specifically, the computer 130 starts at the upper left and lower right pixels of the remaining image 1054 and continues to perform the tests described below pixel by pixel and column by column.

【0080】

Although not obligatory, it is preferred that each dot 1040 occupies every single pixel in the digitized video image. To determine if the current pixel should be dot 1040, the computer 130 accesses and stores a portion of the stored background map in memory that corresponds to the current, the position occupied by the test pixel. Determine if the test pixel should be a dot based on the background map provided. If the stored background map shows that the test pixel is a dot, computer 130 proceeds to decision block 1118, where computer 130 determines if the test pixel is black.

【0081】

If the test pixel is black, or any other hue determined by the software, computer 130 goes to block 1120 and designates the test pixel as the background pixel to be removed later. Computer 130 goes from block 1120 to block 1122, where computer 130 chooses the next pixel in the column, the pixel immediately adjacent to the test pixel.

【0082】

On the other hand, if computer 130 determines in decision block 1118 that the test pixel is not black, computer 130 proceeds to block 1124 and designates the test pixel as the object edge pixel. Computer 130 goes from block 1124 to block 1122 and selects the next pixel.

【0083】

If computer 130 determines in block 1116 that the current pixel should not be a dot, computer 130 proceeds to determination block 1126 to determine if the test pixel is white. If not white, computer 130 goes to block 1124 and designates the test pixel as the object edge pixel. Otherwise, computer 130 goes to block 1120, and goes to block 1122 as described above.

【0084】

Computer 130 proceeds from block 1122 to decision block 1128, where it determines if the new test pixel is later than the last pixel in that column of the remaining digitized image. If the test pixel is not after the last pixel in the column, computer 130 regresses to block 1116. Otherwise, computer 130 proceeds to block 1130, where it moves to the next column of the remaining digitized images and selects the first pixel in that column.

【0085】

Computer 130 proceeds from block 1130 to decision block 1132 to determine if the test pixel is in the last column of the remaining digitized image. If not in the last column, computer 130 regresses to block 1116. Otherwise, computer 130 exits the function of FIG. 9b with function 1132'and returns to block 1114 of FIG.

【0086】

At block 1114, computer 130 removes the portion of image 1054 represented by the pixels designated as the background pixels of block 1112. After the residual background has been removed, image 1954 is represented as shown in the figure specified by Roman numeral III in Figure 8.

【0087】

Computer 130 proceeds to block 1134 in Figure 7. There, an operator of system 1010 (eg model 1022) places an image of an object selected from the memory of computer 130 against background 1133 as desired. This background selection is done by operating Control 1032 correctly so that the operator can select the background of the image he wants to place. Otherwise, the background is automatically selected by the computer 130.

【0088】

Computer 130 travels from block 1134 to block 1136 in Figure 7, where the operator of system 1010 wants an object in the chosen background while looking at the composite image of monitor 1020 and operating control 1032 (Figure 5) correctly. Correct the position on the street. In addition, the operator can place the object in the background as desired, i.e., the object can be superimposed to any part of the background desired by the operator. This position correction is accomplished by a control mechanism (not shown) or automatically by the computer 130.

【0089】

Looking back at FIG. 7, computer 130 proceeds to block 1138, where computer 130 fuzzes the edges of the object as described with respect to FIG. Computer 130 goes from block 1138 to block 1140. At block 1140, computer 130 blends the surrounding background with the edges of object 1022 to sharpen the edges of the object, as represented by Roman numeral IV in FIG.

【0090】

The blending function 1140 is more fully understood with reference to FIG. Computer 130 starts at block 700 in FIG. 19, where it starts with the first row of objects. Computer 130 then proceeds to block 702, where it selects the edge pixels of the object and determines the hue gamma value of the edge pixels. The selection of the edge pixel of the object is made at a software-determined address, such as the upper left edge pixel. Computer 130 proceeds from block 702 to block 704 to determine the hue of the background pixels immediately next to or below the edge pixels of the selected object in the same row as the edge pixels. Computer 130 then proceeds to block 706 to determine the average hue of the two pixels. Computer 130 goes from block 706 to block 708, where computer 130 sets the transparency / opacity attributes of the edge pixels according to the blend factor set in the calling function (Figure 7). The blend factor is in the header of the file. This blend factor, which has a value from opaque to transparent, is used for the dynamically changing transparency of the object, and within this range after trial and error by the programmer as to what kind of result is desirable. It is selected from the value of.

【0091】

Computer 130 goes from block 708 to block 710, where it determines if there are other edges in that row. If so, computer 130 proceeds to block 712, selects the edge pixel immediately next to the pixel whose transparency / opacity has just been set in block 708, and then returns to block 702. If there is no other edge in the column, computer 130 determines in block 714 whether the column is the last column of the object. If so, computer 130 returns to Figure 7. If not in the last column, computer 130 goes to block 716 to get the next edge pixel in the next column. Computer 130 then returns to block 702. If the pixel tested was the last edge pixel to be blended with the background, computer 130 exits the functionality shown in Figure 19 and proceeds to block 1152 in Figure 7.

【0092】

In block 1152 of FIG. 7, the computer 130 changes the hue gamma value of the selected part of the object of the video image and the hue gamma value of the selected part of the background of the composite video image, and the object changes it. On the other hand, the digitally superimposed background and the object appear to be images under the same brightness conditions. After completing the gamma adjustment step for block 1152, computer 130 proceeds to block 1153 to end the background strip process. The details of the process performed by computer 130 in block 1152 are better understood with reference to FIG. A function 266 (FIG. 3c) for matching the gamma of an object or user with the gamma of a selected background scene (FIG. 4e) will be described with reference to FIG.

【0093】

Computer 130 starts at block 266 in FIG. 13 and measures the hue gamma of background 1034 (FIG. 6) at the address determined by the software in the object. The address determined by the software may be a predetermined part such as the imaged person's hand, face or clothes, or may be defined in relative coordinates. Computer 130 proceeds from block 266 to block 400, where it measures the hue gamma of the background at a predetermined, software-determined address.

【0094】

Computer 130 then proceeds to block 402 to measure the hue gamma of object 1022 at a predetermined, software-determined address.

【0095】

Computer 130 proceeds from block 402 to block 404, where the desired gamma is selected. In other words, in block 404, computer 130 selects which lighting conditions are desirable. Computer 130 then proceeds to decision block 406 to change the hue gamma of the object to match the hue gamma of the background, or the hue gamma of the background to match the hue gamma of the object. To decide. If the hue gamma of the object changes, computer 130 goes to block 408 and sets the gamma of the object to be equal to the hue gamma of the background. Conversely, if the background hue gamma changes, computer 130 goes to block 410 to set the background gamma equal to the gamma of the object.

【0096】

Alternatively, the user or operator of system 1010 chooses what lighting system is desirable. This determination by computer 130 can be made in response to interactive commands by the operator of system 1010. That is, the operator of system 1010 wants to change the background illumination conditions to match the illumination conditions imaged under it, as in step 410, or the operator underneath, as in step 408. You can decide if you want to change the visual lighting conditions imaged in in to match the background lighting conditions. Further, if desired, the apparent lighting conditions of the object and the background can both be changed to match the third lighting condition (not shown in FIG. 13).

【0097】

After completing step 408 or 410, computer 130 returns to step 412 in Figure 7 to end the checkerboard background gamma process.

【0098】

In another embodiment, the image includes an object and a background, the background of which is a continuous hue. Next, the function of the main control flow will be described. (The main control flow has been described above in FIG. 3.) FIG. 10 describes the function 234 (FIG. 3b) of stripping the background portion of the image. Function 234 is activated and computer 130 proceeds to step 310 to select the upper left or lower right pixel of the video image 144 passed to the function shown in FIG. 4b. However, the principles of the process performed by computer 130 described below also apply when selecting only one pixel as the starting point, or when selecting three or more pixels as simultaneous starting points. For example, as shown in Figure 4c, computer 130 can start simultaneously at the four corners of image 144 and perform the process described below. The process described below is symbolized in a suitable computer language such as C.

【0099】

Thus, the following process is described for use as a starting point in the upper left pixel of the digitized video image 144 for disclosure purposes. It should be recognized that the principles of the process are similar to those of other starting points, such as the lower right pixel of video image 144, including when computer 130 starts at two or more points at the same time. Control of parallel processes for multiple process areas is known to those of skill in the art.

【0100】

Step 312 shows that computer 130 compares the gamma of the upper left pixel (target pixel) with the gamma of the next pixel in the top row of video image 144 (Figure 4b). As used in the prior art, the gamma of a particular pixel is numeric and refers to one or more data attributes that characterize the pixel. For example, a video image pixel has attributes related to the hue, intensity, brightness, saturation, and contrast of the image portion represented by the pixel. Thus, each such pixel has a numerical "gamma" that represents each of the above-mentioned attributes of that particular pixel.

【0101】

In the present invention, the gamma compared between pixels is the hue gamma of the pixel, but other gammas can also be used. In this embodiment, the hue gamma of each pixel is an integer from 0 to 255, and the hue gamma value indicates the hue of the part of the image represented by the pixels. If the video image is black and white, the hue gamma of the pixel represents the gray shadow of the pixel, or the gray scale value.

【0102】

Therefore, in the case of a continuous hue background 114 (FIG. 4b), adjacent background pixels have substantially the same hue gamma value, and the specific value depends on the color of the background 114. For example, if the background is defined as saturated blue, 255, the background pixels will rarely deviate by more than 20-40. Therefore, the computer 130 infers that the upper left pixel of the video image 144 is the background pixel and uses this pixel as the standard pixel. Computer 130 compares that hue gamma with the hue gamma of adjacent (thus considered test pixels) pixels in the same column as the target pixel, and the adjacent pixels (test pixels) are background pixels as well. Determine if. This step is represented by determination step 314 of FIG.

【0103】

More specifically, as suggested in determination step 314, the computer 130 predetermines the difference between the hue gamma of the pixel in the upper left corner (target pixel) and the hue gamma of the adjacent pixel (test pixel). Compare the differences. If the predetermined difference is greater than the difference between the hue gamma of the two pixels, then the test pixel has approximately the same hue as the target pixel and therefore the background pixel is flagged to work later. This means that the computer 130 proceeds from step 314 of FIG. 10 to judgment step 316. In step 316, computer 130 determines if the test pixel is the last pixel in the column. If there are more pixels in that column, computer 130 proceeds to step 318, where it defines the old test pixel as the new target pixel and selects the next pixel in that column as the new test pixel. Then, the computer 130 proceeds to step 312, determines the difference between the hue gamma of the target pixel and the hue gamma of the test pixel as described above, and compares this difference with the predetermined difference in the determination step 314.

【0104】

On the other hand, if computer 130 determines in decision step 316 that the test pixel is the last pixel in that column, computer 130 goes to step 320 to determine if it has reached the last column in image 144 (Figure 4b). move on. In other words, in decision step 320, if the particular process is part of a process initiated at the pixel in the upper left corner of the digitized image, then the column that has already been tested is digital. Determine if it is the lowest column of the digitized image. On the other hand, if that particular process is part of a process that started at a pixel in the lower right corner of the digitized image, the computer 130 will have the column that has already been tested in the top row of the digitized image. Determine if there is. If not, computer 130 proceeds to step 322, where the target pixel is defined as the last pixel in that column and the test pixel is defined as the first pixel in the next lower column. Computer 130 then returns to step 312 to begin comparing the pixels in the next column. Computer 130 resumes the gamma test comparison described above. Therefore, the computer 130 reads and tests the pixels of the background composition of the video image 144 pixel by pixel and column by column.

【0105】

Computer 130 indicates in decision step 314 that the test pixel is not a background pixel, and thus a representation of the edge 146 of the image of user 112 (FIG. 4c) imaged against a monochrome background 114 (FIG. 1). If the computer 130 determines that the difference between the pixels and the test pixels exceeds a predetermined difference, the computer 130 stores the position of the test pixels in memory. In other words, computer 130 keeps the test pixel on the map as part of edge 146. As shown in FIG. 5, computer 130 proceeds to step 320 and resumes the above process.

【0106】

The computer 130 starts at the pixels in the upper left and lower right corners at the same time and continues the above process until it determines that there are no more pixels to be tested, as indicated by the affirmation in decision step 320. When there are no more pixels to test, computer 130 proceeds to step 324, where it "floods" the background (ie, outside the transparent edge 146) by setting the hue gamma of the background pixels to zero. Turn over all the pixels). In other words, computer 130 removes the portion of image 144 (FIG. 4b) represented by the pixel designated as the background pixel 114 in step 314. Also, each background pixel 114 can be made transparent as soon as the computer 130 determines that a particular pixel is indeed a background pixel. Computer 130 then proceeds to step 326 and returns to the calling program. An overview of the digitized video image 114 during this processing stage is shown in Figure 4d.

【0107】

Therefore, the computer 130 removes only the portion of the video image 144 that is substantially outside the edge 146 (ie, the background component of the image) and the portion of the image 144 that is inside the edge 146 (ie, the image). The entire target component) is left as it is. As a result, the portion inside the edge 146 of the image 144 is not excluded from the image 144 even if it has the same hue as the background 114. Further, in contrast to the chroma key technique, it is no longer necessary to use a specific predetermined hue as the background hue, and any hue can be used as the background hue.

【0108】

From the above disclosure, the predetermined hue gamma difference is selected large enough to prevent edge mapping by background test pixels with hue gamma slightly deviated from the hue gamma of the target pixel. Will be understood. However, on the other hand, the predetermined hue gamma difference allows the presence of the edge pixel of the object to be accurately detected even when the edge pixel has a hue gamma value relatively close to the value of the background hue gamma. Is selected small enough. The exact value of the predetermined hue gamma difference may vary depending on the application, but it may vary depending on the lighting conditions, location, subject, and the like. In the currently preferred embodiment, the predetermined hue gamma difference is 20.

【0109】

With reference to FIG. 11, the function 260 (FIG. 3) for matching the size of the object with the size of the object in the selected background will be described. In a preferred embodiment, the object is the head and neck of the user 112, and the selected background object is the head and neck of the background person 145 (FIG. 4e) to be replaced.

【0110】

Function 260 is started and the computer 130 moves to step 342 of grasping the height of the face of the user 112. Referring to FIG. 4f, a portion of the background scene selected by the user in step 236 (FIG. 3b) is shown, including a background person 145 whose face and neck (shown by the dashed line) will be replaced. Horizon 160 is the coordinate y₀Is centered on the pupil of the eye. Cross symbol 162 is the coordinate y₁Centered on the middle of the neck and at the tip of the chin. Line 164 shows the border of the rim of a garment, eg, a uniform, where the user's neck below it is designated as a lower priority layer than the uniform. Therefore, the uniform may appear to cover the user's neck in the final composite image. The height of the face is the coordinates y₀And y₁Is the absolute difference between. In a background scene where the background person or multiple person selections are given, the face height information for each person in the background is pre-computed and in the file header for the file containing the image of the background scene. It has been saved. The file header format for the preferred embodiment at this time is symbolized and is specific to that embodiment. The header is symbolized by XORing with a pseudo-probability function. The file header contains RGB information for clarifying the characteristics of the face or the object in the file image, the position of the eyes, the position of the chin (cross symbol), the height of the face, and the like.

【0111】

See Figure 4g to show the image of the user's face and neck. Horizon 170 is y₀'It is centered on the pupil of the eye in coordinates. Cross-symbol 172 or other location designation means, y₁'It is centered on the center of the neck and the tip of the chin in coordinates. The height of the user's face is the coordinates y₀'And y₁The absolute difference between'. The user's face height information is stored in the file header for the file containing the image of the user's head and neck.

【0112】

Returning to FIG. 11, after completing step 342, the computer proceeds to step 344. In step 344, computer 130 compares the height of the user's face captured in step 342 with the height of the background person's face at the position selected by the user in step 236 (FIG. 3b). If the height of the user's face is low in the determination of step 346, for example, if the user 112 is a child, the computer moves to step 348. In step 348, computer 130 balances the image of the user's face and neck, expands until the height of the user's face is equal to the height of the person's face in the background, and returns in step 352. If in step 346 the height of the face of user 112 is higher than that of the person in the background, computer 130 moves to step 350. In step 350, the computer 130 balances the image of the user's face and neck, shrinks or enlarges until the height of the user's face is equal to the height of the background person's face, and returns in step 352. .. However, if in decision step 346 the height of the face of user 112 is equal to that of the person in the background, computer 130 returns to calling main flow 200 (FIG. 3) in step 352.

【0113】

With reference to FIG. 12, a function 264 (FIG. 3c) for matching the object, that is, the skin of the user, with the skin of the person 145 (FIG. 4e) in the selected background will be described. Function 264 starts and proceeds to step 370 instructed to measure the area of the background person to be replaced in the composite image. Moving on to step 372, computer 130 measures three gamma attributes in the designated area. That is, overall hue, overall saturation, and overall intensity. Next, in step 374, computer 130 specifies the area of the user's skin to be measured. The computer 130 knows by line 170 where the user 112's eyes are on the image shown in FIG. 4g. The computer specifies an area of the user 112's image, just above the forehead. This region is then measured in step 376 for the same attributes: hue, saturation and intensity. Moving on to step 378, computer 130 then determines the skin area of the user's facial image to be compared when calling strip function 234, using the attributes that describe the user's skin measured in step 376. To do. Function 234 flags the skin pixels of the user's facial image in this call. Hair, eyebrows, eyes, lips, beard and / or mustache (if any), etc. are not flagged. Then, returning from function 234, computer 130 proceeds to step 10382. In step 10382, the flagged skin pixels identified by function 234 are set to the values of the attributes measured in step 372 of the background person's skin, then function 264 is set to the calling main flow 200 in step 10384 ( Return to Figure 3).

【0114】

A function 266 (FIG. 3c) for matching an object or user gamma to the gamma of a selected background scene (FIG. 4e) will be described with reference to FIG. Function 266 makes it possible to express the object as if it were imaged under the same lighting conditions as the background in which the object is digitally stacked. Function 266 begins and the computer 130 proceeds to step 400 to determine the type of lighting used for the background scene (FIG. 4e). Illumination conditions are determined by measuring the gamma attributes of hue, saturation, and contrast in the area of the background scene determined by software. Moving on to step 402, computer 130 makes the same gamma attribute measurements as in step 400, this time on user image 144 (FIG. 4d).

【0115】

The computer 130 then proceeds to step 404 where the computer 130 selects the desired lighting conditions. In other words, in step 404, the computer 130 determines whether to change the hue gamma of the object to match the hue gamma of the background or to change the hue gamma of the background to match the hue gamma of the object. To do. This determination by computer 130 can be achieved in response to interactive commands by the user of system 100. That is, the System 100 user wants to change the lighting conditions of the background scene to match the lighting conditions that the user imagined underneath, or the lighting conditions that the user imagines underneath of the background scene. You can decide if you want to change it to match the lighting conditions. In addition, the apparent lighting conditions of the object and the background may both be modified to match the third lighting condition, if desired (this choice is not shown in FIG. 8).

【0116】

From step 404, computer 130 proceeds to decision step 406 to determine whether to change the object gamma. If so, computer 130 proceeds to step 408 so that the gamma attribute values for the hue, saturation, and contrast of the object are equivalent to the gamma attribute values for the hue, saturation, and contrast of the background scene. Set to. Otherwise, computer 130 proceeds to step 410 and sets the gamma value of the background scene to be equal to the gamma value of the object. After completing either step 408 or step 410, computer 130 proceeds to step 412 and returns to the main calling flow 200.

【0117】

A function 268 for enhancing pixels in an image will be described with reference to FIG. Function 268 is called by either main flow 200 (Fig. 3C) or top-level gamma function 262 (Fig. 17). Function 268 is activated and proceeds to step 420, where computer 130 begins processing the image at a predetermined location, eg, X, Y coordinates 0,0. Going to step 422, computer 130 determines the hue value of each pixel in the current column and attempts to find the edge of the object in that column. If an edge is found, computer 130 proceeds to step 426, as determined by decision step 424. The edge is found if the computer 130 determines that the hue of the pixel has a significant change compared to the previous pixel. In a preferred embodiment, if the maximum range of hue values is 0 to 255, such changes will be found if the hue values vary by 80 or more. Of course, the change threshold can be any value within the above range selected for the particular application of this system.

【0118】

In step 426, computer 130 selects three pixels just outside the edge of the object and uses them as edge pixels when performing the blending operation. The blending operation is performed by computer 130 in steps 428, 430, and 432, where the three pixels mentioned above are blended into the background layer just below the current object layer. In step 428, for each of the three pixels, the computer 130 determines the hue of the background pixel in the layer below the object pixel, and the hue value of each of the three background pixels is the corresponding object in step 430. Average in object pixels. Going to step 432, the computer 130 sets the transparency or opacity of each of the edge pixels of the three object layers according to the blend factor in the file header. The blend factor, which has a value from opaque to transparent, is used for the dynamically changing transparency of the object, and is selected from the values within that range by a programmer who makes trial and error from the viewpoint of what kind of result is desirable. Will be done. After step 432 is completed, the computer moves to step 434, where computer 130 fuzzes two pixels inside the edge of the object using the edge found on the current column according to the decision in step 422. Choose for. The fuzz operation is performed by computer 130 in steps 436, 438, and 440 to smooth the edges. In step 436, for each of the two pixels, the computer 130 determines the hue of the three pixels in close proximity to them on the same column. The computer 130 then determines the average hue value between the three pixels, as shown in step 438. Moving on to step 440, the computer 130 sets the hue value of each of the above two pixels to be equal to the mean value determined in the previous step 438.

【0119】

After step 440 is complete, the computer returns to step 422 and attempts to find another edge in the current column. If another edge is found, the process described above is carried out according to the determination in determination step 424. However, if the edge of the current column is reached without finding an edge, computer 130 moves to decision step 1302 to determine if the last column of the image has just been processed. If not, computer 130 proceeds to step 1304, selects the next column to process, and returns to step 422 to search for edges. However, if the last column has been processed, the computer calls the gradient sharpening function 1306 according to the decision in decision step 1302 to emphasize the edges of the object.

【0120】

The file header has a predetermined value for the degree of sharpening performed. For example, an image cannot be sharpened on the edges, but can be sharpened fully at the center, that is, 100% on a linear scale between the center and the boundary. Other percentages can also be used as sharpening factors. For example, an image can be sharpened between the center and the border on a linear scale by 30% at the left edge, 20% at the center, and 80% at the right edge. Function 1306 will be described later. After the function 1306 returns to the function 268, the computer 130 exits the pixel enhancement function 268 in step 1308.

【0121】

The details of the processing performed by the computer 130 in function 270 of FIG. 3c are more fully understood with reference to FIG. The function 270 for arranging the object or the user 112 in the selected background scene (Fig. 4e) will be described. Function 270 begins and computer 130 proceeds to step 460 to obtain the address of cross-symbol 172 (Fig. 4g) on the user image. This symbol is centered on the neck and below the chin on the user image, as placed by user 112 in step 258 of FIG. 3c. The address of the cross symbol 172 is searched from the file header of the user image file. Computer 130 then proceeds to step 462 to find the address of the cross symbol 162 (Fig. 4f) for the selected person 145 in the background scene. This address is searched from the file header of the background scene image file. The computer then sets the address of the user's cross-symbol 172 to be equal to the address of the cross-symbol 162 of the selected person 145 in the background scene. Therefore, when the composite images are overlaid together, the user 112's image is in the correct position. The computer 130 returns to the calling main flow 200 (Fig. 3) in step 464.

【0122】

The details of the processing performed by the computer 130 in function 274 of FIG. 3c are more fully understood by reference to FIG. The function 274 for overlaying an object or personalized text in a selected background scene (FIG. 4e) will be described. Function 274 is called twice by the main control flow 200. In a preferred embodiment, the first call is to overlay the image of the user's head and neck so that the user image fits into, for example, a shirt or uniform in the final composite image. The shirt takes precedence over the user image so that the user's neck and head are visible at the top of the shirt. Both the shirt and the user's image take precedence over the background scene.

【0123】

In a preferred embodiment, the second call is to overlay the personalized text entered by the user in step 244 (FIG. 3b). The text has the highest priority and is therefore always visible in the final composite image.

【0124】

Function 274 starts and proceeds to decision step 480. Therefore, it is determined whether this routine is a functional call as in the preferred embodiment or a user call as in the alternative embodiment. In a preferred embodiment, computer 130 proceeds to step 482, where an item, such as a text or object, or an area of the image to be overlaid is identified in the argument for the functional call. Computer 130 then accesses the information corresponding to the item or area and proceeds to step 488. However, if in step 480 the call to this routine is a user call, computer 130 proceeds to step 484. In step 484, the area or object of the background scene may be specified by the user 112 to move to another layer. In step 486, the selected area or object is stored in memory with information about the name, layer, position within the layer, and the alpha-bit flag used to set the transparency bits. ..

【0125】

After either step 486 or 482 is complete, the name, layer, location within the layer, and alpha bit flag corresponding to the item or region are linked by computer 130 in step 488. It is written to list). The alpha bit flag indicates the transparency or opacity of the item or area. The position is saved in X and Y coordinates. After the completion of step 488, computer 130 moves on to determination step 490 to check if an option to sew or join together multiple objects on the same layer is called in an alternative embodiment. If so, in option step 492, the computer sutures the desired objects on the same layer into the layer assembly. After the option step 492 is completed, or if the decision step 490 is negative, the computer 130 moves to step 494. In step 494, computer 130 calls the linked list to find the transparent alpha bit flag. In step 496, for the item indicated by the linked list, computer 130 displays on monitor 110 (FIG. 2) an object or area defined by name, layer, and position within that layer.

【0126】

In the alternative embodiment, in option determination step 498, it is determined whether user 112 (FIG. 1) wants to reset the transparent alpha bit flag. If so, the computer goes to option step 500 and the transparency flag is reset to opaque by using a pointing means such as a mouse or by defining a region using X, Y Cartesian coordinates. After step option 500 is complete, or if step 498 is negative, computer 130 moves to option determination step 502. In an alternative embodiment, step 502 determines that the user 112 wants to suture an object or plurality of objects on one layer with a background. If so, in option step 504, the computer 130 sutures the object together with the background into the aggregate on one layer, and the object is removed from the linked list. The aggregate may include objects, areas, backgrounds, text, and the like. In a currently preferred embodiment, the file format for storing the aggregate with the background or as a new background has a file extension ".spo". After the option step 504 is completed, or if the determination step 502 is negative, the computer 130 returns to the calling main flow 200 (FIG. 3) in step 506.

【0127】

The top-level gamma function 262 (Fig. 3c) is described below with reference to FIGS. 17a and 17b. The preferred examples disclosed herein do not include the top-level gamma function 262, but alternative examples may include this function. Function 262 can be used in two ways. That is, as a function call in which the attribute to be changed is selected in advance, or as a user call in which the attribute to be changed is selected by the user 112. If feature 262 is accessed by a user call, System 100 acts as a highly interactive and extremely powerful image editing tool.

【0128】

Function 262 starts and moves to determination step 514 to determine if routine 262 is called as a function. If so, computer 130 proceeds to step 516 to access preset gamma attributes, layers, and object or region selection results, such as all pixels on the layer with a blue phase of values between 10 and 75. To do. Computer 130 then proceeds from step 516 through the off-page connector C580 and resumes at step 582 in Figure 17b. However, if routine 262 is determined to be a user call in step 514, computer 130 proceeds to decision step 520 to check to determine if a strip operation should be performed. The strip operation in this function can be performed on an object or a background scene. If so, computer 130 proceeds to step 522, where user 112 selects the gamma attribute with the corresponding value and / or the layer and object, or the region to be changed on that layer. When step 522 is complete, computer 130 proceeds to step 524 and puts the selection results from step 522 into the matrix. In decision step 526, computer 130 checks if user 112 wants to undo the operation selected in step 520. If so, at step 528, computer 130 cancels the operation selected in step 520.

【0129】

The X-axis of the above matrix contains the gamma attribute. That is, hue (H), saturation (S), lightness (L), intensity (I), contrast (C), red (R), green (G), blue (B), and HSL, HSI, HSC, And contains a combination of RGB. The Y-axis of the matrix contains operations performed by the top-level gamma function. That is, it includes strips, fuzz, blends, gamma changes, transparency / opacity, and pixel enhancement. The matrix is associated with a Z-axis that corresponds to the number of layers in the composite image. In the currently preferred embodiment, the Z-axis utilizes 32 layers. The matrix allows any combination of one or more operations on one or more gamma attributes performed on one or more layers of a composite image.

【0130】

After the completion of step 528, or if the result of decision step 526 is negative, or if the result of decision step 520 is negative, that is, if no strip is selected, computer 130 will use decision step 530 A check is made to determine if a fuzz operation will be performed. Fuzz operations can be performed on an object or in a background scene. If so, the computer proceeds to a set of steps that are essentially the same as steps 522-528 described above, and then to decision step 540. If the decision to perform the operation is unsuccessful in step 530 (and steps 540, 550, and 560 below), the computer 130 performs the next decision step to test for the next possible operation. Proceed to.

【0131】

At step 540, computer 130 performs a check to determine if a blending operation is to be performed. The blending operation in this function can be performed on an object on the background scene or on a nearby layer. If so, the computer proceeds to one set of steps, which is essentially the same as steps 522-528 described above, and then to decision step 550.

【0132】

At step 550, computer 130 performs a check to determine if a gamma change operation is to be performed. Gamma operations in this function can be performed on objects and / or background scenes. If so, the computer proceeds to one set of steps, which is essentially the same as steps 522-528 described above, and then to decision step 560.

【0133】

In step 560, computer 130 performs a check to determine whether to perform a transparent / opaque operation. The transparent / opaque operation in this function can be performed on an object on the background scene or on a nearby layer. If so, the computer proceeds to one set of steps, which is essentially the same as steps 522-528 described above, and then to decision step 570.

【0134】

In step 570, computer 130 performs a check to determine if a pixel enhancement operation will be performed. Pixel enhancement operations in this function can be performed on objects or background scenes. If so, the computer moves to one set of steps, which is essentially the same as steps 522-528 described above, and then to step 582 via the off-page connector C580 in Figure 17b. If the pixel enhancement operation is not selected in step 570, the computer 130 also proceeds to decision step 582 via the off-page connector C580.

【0135】

The next three steps (582, 584, 586) of function 262 are preset operations (from step 516) or elected operations (from steps 520 to 570) in preparation for step 590 by computer 130. It is carried out continuously for each of. At step 582, computer 130 has a layer limit address. Do test). In the currently preferred embodiment, there are 32 layers. Computer 130 tests whether it is in the top layer (Z = 31) or the bottom layer (Z = 0). Moving on to step 584, computer 130 prioritizes the layers and operations. The first preferred operation is transparent / opaque. Layer zero (Z = 0) is the original background scene. As an example, to make the region above layer zero opaque, nothing must be opaque in the same region of 31 layers above layer zero, so the region is transparent in each of the 31 layers. Is set to. As another example, if layer 7 (Z = 7) is translucent, then all layers above layer 7 (Z = 8-31) must be transparent and the layers below layer 7 (Z = 8 ~ 31) must be transparent. The transparency / opacity of Z = 0 to 6) is adjusted so as to correspond to the translucency of layer 7. In another example, the top layer (Z = 31) is opaque, so no changes are needed in the other 31 layers. Other preset or selected operations are then prioritized. In step 586, computer 130 seeks out the affected layer for each of the preset or selected operations. There can be six different layers in the priority list that correspond to the six operations from steps 520 to 570. Steps 582 to 586 are continuously performed until the layer limit address test is completed for each operation.

【0136】

In step 590, computer 130 performs a layer-by-layer process for scheduling the execution of the operation selected in steps 520-570 or 516 as described above. In determination step 592, computer 130 checks whether the operation scheduled to be performed is a strip operation. If so, in step 594, computer 130 (selected to be performed by step 590), using the method of starting at the four corners and advancing through the rows, as described above in relation to feature 234. Perform the strip on the layer. If the test on step 592 is unsuccessful (not stripped), then an operation scheduled by computer 130, such as fuzz, is chosen to be performed by a column-by-column method (step 590). It is executed on the layer. After all columns of the layer have been processed in step 596, computer 130 proceeds to determination step 598 to check if further layers are processed. If so, computer 130 regresses to step 590 and schedules the next selected layer for the same operations that were previously performed, such as fuzz operations. After one operation has been performed on all desired layers, the decision step 600 is made to determine if additional operations will be performed. If so, computer 130 regresses to step 590 and schedules the execution of the next elected new operation, such as blending. If all operations are completed in step 600, the computer returns to caller flow 200 (Figure 3) in step 602.

【0137】

Top-level gamma function 262 can be used to create interesting composite images. As an example, the user wants to replace the image of his head with the image of the head of a hockey player on a hockey competition card. The hockey player's face has a large scar that the user wants to be in the image of his face in the final composite image. Top-level gamma function 262 can be used to isolate and opaque the image of the scar. The rest of the image of the hockey player's head is transparent. The image of the user's head should be opaque except for areas with scars that should be transparent. In the resulting composite image competition card, the scar appears on the image of the user's head on the image of the hockey player's body.

【0138】

Perform the operation so that the four functions are selected by step 530, 540, 550 or 560 of the top level gamma function 262 (Figure 17). The actual schedules and calls for performing these functions are performed by steps 590 and 596. However, for reference, the operation selected by step 530 is referred to as the fuzz function 530', the operation selected by step 540 is referred to as the blend function 540', and the operation selected by step 550 is referred to as the gamma change function 550'. And the operation selected by step 560 is referred to as the transparent / opaque function 560'. These four functions are described below. The function selected by step 520 is performed by strip function 234 as described in relation to FIG. The function selected by step 570 is performed by the pixel enhancement function 268 as described in relation to FIG.

【0139】

After the background component 114 of the digital image 144 (Figure 4b) is effectively removed, the component representing the image of the user 112 is maintained (Figure 4b) and is characterized by a pixel width of approximately one or two pixels. It has a relatively "fuzzy" edge. Therefore, in order to create a digital image with smooth edges, the digital computer 130 performs the fuzz function 530'as shown in FIG. Fuzz function 530'can be called by either pixel enhancement function 268 (Fig. 14) or top level gamma function 262 (Fig. 17). The function begins at step 530'and proceeds to step 680. The computer 130 then selects one of the edge pixels (ie the "test" pixel) using a software-determined address (eg by selecting the top left pixel) and determines its hue in step 682. .. The computer 130 then selects the edge pixels in the immediate vicinity of the test pixels with respect to the 3 pixel spacing on the row and determines the hue of these pixels, as shown in step 684.

【0140】

The computer 130 then determines the average hue gamma value of the three pixels, as shown in step 686. As shown in step 688, computer 130 then sets the hue gamma value of the test pixel to be equivalent to the mean value calculated in step 686. Computer 130 proceeds to determination step 690 to determine if the test pixel is the final edge pixel processed by the fuzz function 530'. If not, computer 130 proceeds to step 692, selects one edge pixel in the immediate vicinity of the test pixel, designates this nearby pixel as the new test pixel, and returns to step 682. If in step 690 the computer 130 determines that the test pixel was the last edge pixel, the computer 130 exits the fuzz function 530'in step 694.

【0141】

In function 540', computer 130 blends the surrounding background with the edges of an object or area on a layer close to the background scene layer. The blending function 540'is better understood by referring to FIG. Blend function 540'can be called by either pixel enhancement function 268 (Fig. 14) or top level gamma function 262 (Fig. 17). Computer 130 starts at step 540'and proceeds to step 700 to select the edge pixel of the object as the test pixel. The selection of edge pixels for an object is made at a software-determined address, such as the left edge pixel in the first column of the object. Going to step 702, computer 130 determines the hue gamma value of the test pixel. From step 702, computer 130 proceeds to step 704 to determine the hue of the background pixel above the layer immediately below the selected test pixel in the same column as the test pixel. Computer 130 then proceeds to step 706 to determine the average hue of the two pixels. From step 706, computer 130 proceeds to step 708 to set the transparency / opacity of the test pixels according to the blend factor in the calling function.

【0142】

From step 708, computer 130 proceeds to decision step 710. Computer 130 then determines if the current column has other edges, i.e., the test pixel is the last edge pixel processed in the current column. If at least one additional edge is in the current column, the computer proceeds to step 712 to select the next edge pixel. This pixel is designated as the new test pixel and computer 130 returns to step 702. However, if the determination in step 710 means that there are no more edge pixels in the current column, computer 130 proceeds to determination step 714 to perform a check to determine if the last column of the object has already been processed. If not, computer 130 proceeds to step 716 to select the next new column and the first edge pixel in that column. This pixel is designated as a new test pixel, and computer 130 returns to step 702. If in step 714 the computer 130 determines that the last row of objects has just been processed, the computer 130 exits the blend function 540'in step 718.

【0143】

With reference to FIG. 20, the function 550'that changes the gamma attribute of the image will be described. Function 550'is called by top-level gamma function 262 (Figure 17). A list of gamma attributes that can be modified is disclosed in connection with Figure 17. File headers can contain information about regions or objects where gamma attributes or multiple attributes are changed. Depending on the complexity of the background, the header may contain additional information necessary to properly achieve the reality of the photograph. This gamma change feature allows the user to change one or more gamma attributes in the entire layer or in regions or objects on the layer. For example, the saturation attribute can be pumped up to a value between 40 and 80 in the entire layer.

【0144】

Function 550'starts and proceeds to step 730, where computer 130 begins processing at the position of the image determined by the software. This position can be, in some embodiments, the X, Y coordinates of the top left corner of the object or region, or the entire layer. Going to step 732, computer 130 changes the pixel value of the selected attribute, eg hue. Computer 130 then moves on to determination step 734 to determine if the additional preselected pixels (of the object, region or layer) in the current column have modified attributes. If so, computer 130 moves to step 736 and advances to the next pixel in the current column. The computer then returns to step 732. If the determination in step 734 does not have an additional preselected pixel in the current column, computer 130 moves to determination step 738 to determine if the last column of the selected area has already been processed. If not, computer 130 moves to step 740, advances to the first pixel on the next column of the selected area, and returns to step 732. However, if the selected area, object, or last column of layer has already been processed, the computer moves to decision step 742 to determine if there are other attributes to change. If so, computer 130 goes to step 7130, gets the next gamma attribute to change, and then goes back to step 730. If, at the discretion of step 742, there are no more attributes to change, the computer 130 exits the gamma change function 550'in step 746.

【0145】

The transparent / opaque function 560'will be described with reference to FIG. Function 560'is called by top-level gamma function 262 (Figure 17). Function 560'starts and proceeds to decision step 770. Therefore, the computer 130 determines whether the function is performed on the object or the background scene. If it is determined in step 770 that the object is to be processed, computer 130 proceeds to step 772 and begins processing in the first column of the object identified by the file header. Computer 130 selects the edge pixel of the object on the current column as the test pixel. The selection of the edge pixels of the object is made at a software-determined address, eg, the left edge pixel of the first column of the object. Going to step 774, computer 130 determines the transparency / opacity (T / O) value of the test pixel. From step 774, computer 130 proceeds to step 776 to determine the T / O value of the background pixel on the layer immediately below the test pixel in the same column as the selected test pixel. Computer 130 then proceeds to step 778 to determine the average T / O of the two pixels. From step 778, computer 130 proceeds to step 780 to set the transparency / opacity of the test pixels according to the blend factor in the calling function.

【0146】

From step 780, computer 130 proceeds to determination step 782 to determine if there are other edges in the current column, i.e. the test pixel is the last edge pixel processed in the current column. If there is at least one additional edge pixel in the current column, the computer moves to step 784 and selects the next edge pixel. This pixel is designated as a new test pixel, and computer 130 then returns to step 774. However, if the determination in step 782 means that the edge pixels are no longer in the current column, computer 130 moves on to determination step 786 to determine if the last column of the object has already been processed. If not, computer 130 proceeds to step 778 to select the next column and the first edge pixel in that column. This pixel is designated as a new test pixel, and then the computer returns to step 774. If in step 786 the computer 130 determines that the last row of objects has just been processed, the computer 130 moves to step 790. In step 790, computer 130 initiates further processing by using the strip function 234 (FIG. 10) to access the first pixel in the first column of the transparent area.

【0147】

If the decision in decision step 770 is that feature 560'is processing a background scene, computer 130 moves on to step 800 to access the first pixel in the first column of the image as a test pixel. .. After completing step 800, computer 130 moves on to step 802 and uses the file headers to determine whether the hue, red, green, blue (RGB), and intensity tests are performed. The tests may be performed in any combination, or all five tests may be performed. The test determines the hue, red, green, blue, or intensity values of a pixel. For each test, the range is stored in the file header, along with information indicating whether the test for the test pixel is for values inside or outside the range. Range and inside / outside information can be different in each of the five tests. As an example, a predetermined hue value limit can be from zero to 255, and the test may look for pixels outside the hue range of 50 to 100. These pixels will have their own alpha bit flag set. Therefore, of steps 804, 808, 812, 816, and 820, each range will be checked only for the tests selected to run according to the file headers. Therefore, in the above example, the computer 130 moves to determination step 804 to determine if the hue value of the test pixel is outside the range specified in the header, eg 50-100. If so, computer 130 sets the alpha bit flag transparent in step 806. However, if in step 804 it is determined that the pixel values do not match the range specified for hue in the file header, computer 130 will take steps 808, 812, 816, and 820, if any. Moves to the next decision block selected in the file header. A set of steps similar to steps 804 and 806 above is a file

【0148】

After completing one to five tests above, computer 130 moves to decision step 824 to determine if at least one additional pixel is in the current column to be tested. If so, computer 130 proceeds to step 826, accesses the next pixel in the current column, and returns to step 802 to repeat the process described above. If the determination in step 824 has processed the last pixel on the current column, computer 130 moves on to determination step 828 to determine if the last column has just been processed. If not, computer 130 moves to step 830, advances to the next column of images, and returns to step 830. If the final column has already been processed at the discretion of step 828, the computer 130 exits the transparent / opaque function 560'at step 832.

【0149】

After completing step 790, computer 130 moves to step 802. Steps 802-832 are similar to the steps described above, except that the alpha bit flag is set to opaque rather than transparent for steps 806, 810, 814, 818, and 822. This is because the area to be processed (when step 802 is reached through step 790) is already transparent. As an example, setting the alpha bit opaque allows the user to create a drop shadow in the shape of the user's head, known as shading protection.

【0150】

The inclination sharpening function 446 will be described with reference to FIG. 22. This function 446 is called by the pixel enhancement function 268 (Fig. 14). As an example, function 446 can be used to look for eyes on the image of the face so that the eyes can be emphasized without changing the skin. Function 446 starts, computer 130 proceeds to step 850, and accesses the first pixel in the first column as the test pixel of the image passed from function 268 to function 446. Moving on to step 852, the computer 130 measures the hue of the test pixel and proceeds to judgment step 854. In step 854, the computer determines if the hue value is outside the range specified by the file header, eg 110-150. If so, for example, if the value is less than 110 or greater than 150, computer 130 moves to step 856 and the alpha bit flag is set for that pixel. After the completion of step 854, or if the judgment of step 854 indicates that the hue value is within the range, the computer 130 moves to the judgment step 858. In step 858, the computer determines if at least one additional pixel remains in the current column. If so, computer 130 moves to step 860, accesses the next pixel in the current column, and returns to step 852. However, if computer 130 determines in step 858 that the last pixel in the current column has already been processed, determination step 862 determines if the last column has just been processed. If not, computer 130 goes to step 864, accesses the next column, and returns to step 852 to process the new column.

【0151】

If in step 862 it is determined that the last column has already been processed, computer 130 proceeds to step 870 to find the first column with pixels that have the alpha bit flag set. Moving on to step 872, the computer 130 performs edge enhancement on the pixels. The computer 130 performs one, two, or three operations on the pixel with the hue value of the pixel and the accompanying set of ranges for the hue value and the associated operations stored in the file headers. The three operations are to saturate the pixels, shift the hue (make black blacker, make white whiter), and shift the intensity. A set of hue value ranges is placed in the file header, for example, if the measured hue value exceeds the file header value by 40 or more, a saturation operation is performed, and if the measured value exceeds the header value by 70 or more, a saturation operation is performed. A shift intensity operation is performed, and if the measured value exceeds the file header value by 100 or more, the saturation and shift hue operations are set to be performed. The hue range is specific to the background and is set by the programmer to achieve the desired result based on the background. For example, if the skin color of the object in the background is green, this range is set so that the skin color of the object placed in the background also matches this green.

【0152】

Going to determination step 874, computer 130 determines if there is at least one more pixel with the alpha bit flag set in the current column. If so, computer 130 proceeds to step 876, accesses the next pixel in the current column with the alpha bit flag, and returns to step 872. However, if the determination step 874 is negative, the computer 130 moves on to the determination step 878 to determine if the last column of the image has already been processed. If not, computer 130 goes to step 880, accesses the next column with pixels with alpha bit sets, and returns to step 872. If the final row has already been processed at the discretion of step 878, the tilt sharpening function 446 returns to the pixel enhancement function 268 in step 882.

【0153】

The software mentioned here is written in "C" language, which was formerly Practically Perfect Productions, Inc. of San Diego, California, and is now ImageWare Software Inc. ) Is used to translate the source code into machine-readable object code using the Photo Novelty Programming Language (PNPL) compiler. However, those skilled in the art will appreciate that the steps in the attached flow diagram can be performed using many different compilers and / or programming languages.

【0154】

The image editing system described here has many uses in many environments and can be easily adapted to its use. For example, this system can be used to create composite photographs, for example, in which a portion of an image of a person on a sports team photograph can be replaced with a portion of the image of another person. It can also be used to create postcards that depict a person as if they were in a foreign land. It can also be used in applications such as the production of driver licenses or security access cards that combine the image of a person with other information selected on the card. Therefore, the scope of claims is construed to be within the scope of these and other applications of the present invention, and is not limited to the examples disclosed herein.

【0155】

The above detailed description has shown, described, and pointed out the basic and novel features of the invention that apply to various examples, but various omissions, substitutions, and details of the described inventions have been omitted, substituted, and pointed out. It will be appreciated that changes can be made by one of ordinary skill in the art without departing from the spirit of the invention.

[Simple explanation of drawings]

FIG. 1 is a perspective view of a preferred embodiment of the image system of the present invention, in which a person is seated in front of a partially cut and phantomly represented background with a single hue.

FIG. 2 is a block diagram showing components of the image system of FIG.

FIG. 3a is a top-level flow diagram of the image system of FIG. 1 used on a single color background.

FIG. 3b is a top-level flow diagram of the image system of FIG. 1 used on a single color background.

FIG. 3c is a top-level flow diagram of the image system of FIG. 1 used on a single color background.

FIG. 4a provides a sequence of examples of images obtained by performing the process of FIG. 3 in an image system.

FIG. 4b shows a sequence of examples of images obtained by performing the process of FIG. 3 in an image system.

FIG. 4c provides a sequence of examples of images obtained by performing the process of FIG. 3 in an image system.

FIG. 4d shows a sequence of examples of images obtained by performing the process of FIG. 3 in an image system.

FIG. 4e shows a sequence of examples of images obtained by performing the process of FIG. 3 in an image system.

[Fig. 4f] An example of an image obtained by performing the process of Fig. 3 in an image system is shown in a sequence.

[Fig. 4g] An example of an image obtained by performing the process of Fig. 3 in an image system is shown in a sequence.

FIG. 4h provides a sequence of examples of images obtained by performing the process of FIG. 3 in an image system.

FIG. 5 is a perspective view showing a device of an image system, in which a person stands in front of a background that is partially cut and phantomly represented and has a single hue.

FIG. 6 is another embodiment of the system of the present invention, which is a perspective view showing an electrical image device operated by a coin, has a grid-like background, and is partially removed for clarity. ..

FIG. 7 is a block diagram showing the overall process of the system in the present invention when the grid-like background of FIG. 6 is used.

8a is a conceptual diagram graphically showing video images at various stages during the process of FIG. 7. FIG.

8b is a conceptual diagram graphically showing video images at various stages during the process of FIG. 7. FIG.

8c is a conceptual diagram graphically showing video images at various stages during the process of FIG. 7. FIG.

8d is a conceptual diagram graphically showing video images at various stages during the process of FIG. 7. FIG.

FIG. 9a is a flow diagram showing a function of fuzzing (blurring) the edges of the object shown in FIG. 7.

9b is a flow diagram for performing a coarse function and a precision function for removing a part from a grid-like background in the control flow of FIG. 7. FIG.

FIG. 10 is a flow diagram for performing a strip (removal) function as defined by Glock 234 in FIG.

FIG. 11 is a flow diagram for performing a dimension matching function defined by Glock 260 in FIG.

FIG. 12 is a flow diagram for performing a skin matching function as defined by Glock 264 in FIG.

FIG. 13 is a flow diagram for performing a background gamma function defined by Glock 266 in FIG.

FIG. 14 is a flow diagram for performing a pixel enhancement function defined by Glock 268 in FIG.

FIG. 15 is a flow diagram for performing an alignment function defined by Glock 270 in FIG.

FIG. 16 is a flow diagram for performing a stacking function as defined by Glock 274 in FIG.

FIG. 17a is a flow diagram for performing the highest level gamma function as defined by Glock 262 in FIG.

FIG. 17b is a flow diagram for performing the highest level gamma function as defined by Glock 262 in FIG.

FIG. 18 is a flow diagram for performing a fuzz function corresponding to the Glock 530 of FIG.

FIG. 19 is a flow diagram for performing a mixing function corresponding to the Glock 540 of FIG.

FIG. 20 is a flow diagram for performing a gamma changing function corresponding to the Glock 550 of FIG.

FIG. 21 is a flow diagram for performing a transparent opaque function corresponding to the Glock 560 of FIG.

FIG. 22 is a flow diagram for performing a tilt sharpening function corresponding to the Glock 446 of FIG.

[Explanation of symbols]

106: Camera 110: Video Monitor 116: Payment Adapter 118: Printer 130: Computer 132: Graphic Interface 134: Storage