Image processing apparatus and image processing method

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus and method which can more realistically display reflected light such as diffused light and specular light in relation to a light source; the present invention also relates to a recording medium for storing a program which has a computer execute an image processing procedure.

2. Description of the Related Art

Computer simulations and computer games require image processing, for generating image data for a display screen according to the input data provided, to generate more realistic images in real time. Before now, image processing comprised the following: finding coordinates for flat, multi-sided figures or polygons within a three-dimensional space according to the input data and the like; carrying out perspective conversion to two dimensional coordinates of the display screen, for polygons located within the display screen; and generating color data for each pixel in the polygon.

In such an instance, the polygon data used comprises vertex data, such as color or brightness data, normal vectors, and the three-dimensional coordinates of the vertexes of the polygon. With the vertex data based on a two-dimensional coordinate plane, a rendering process generates color data by performing interpolation (linear interpolation) according to a linear function based on the vertex data for each pixel in the polygon.

In order for the display of more realistic images, the effects of the light source must be reflected in the image. Light sources include point light sources, such as sunlight, with very high brightness; light sources such as a spotlight or laser light source, having directionality and high brightness in that direction; and light sources such as starlight, with low brightness, but with directionality and which are reflected by a mirror surface or the like. Light from such light sources is reflected by an object. This reflection can be classified as diffused reflection or specular reflection. The diffused light from diffused reflection is reflected uniformly in all directions by an object. On the other hand, the specular light from specular reflection is the reflection of light with directionality and is reflected with directionality in a direction wherein the angle of reflection is equal to the angle of incidence. Diffused light, such as the midday sun, includes diffused light due to direct illumination from the sun and ambient light or environmental light from light reflected off surrounding objects. Because environmental light consists of innumerable light sources, environmental light is generally processed as a constant term for diffused light in image processing using computers.

The aforementioned diffused light and specular light are characterized by high brightness within a specified area of illumination and decreasing brightness at the periphery of that area. For example, in a field illuminated by a spotlight, only a limited area is an area with high brightness. Specular light from a light source reflected by a mirror surface likewise has high brightness in a limited area.

The color data of the vertex data composing the aforementioned polygon data must be generated with consideration given to the object's color, as well as brightness due to the aforementioned reflected light, and the color of the light itself.

However, the rendering process in image processing using polygons generally uses the Gouraud shading method, using linear interpolation of vertex data. For this reason, effective lighting processing is not possible, depending on the positional relationship of that area and the polygon, in the case of high brightness in a uniform area such as with the diffused light and specular light.

FIG. 12 is a drawing to explain the problems with the Gouraud shading method. In this figure, a spotlight <b>10</b> directly illuminates one portion of a triangular polygon <b>1</b>, comprising vertexes A, B, and C. Consequently, diffused light is generated in this area <b>10</b>. For this reason, lighting processing reflecting the color of the spotlight and the high brightness must be carried out for this area <b>10</b>. However, none of the vertexes of the polygon <b>1</b> is located within the area <b>10</b> and as a result, nothing is reflected in the vertex brightness and color data. The diffused light of the area <b>10</b> is therefore not reflected in the polygon <b>1</b>.

FIG. 13 is a drawing to explain further problems with the Gouraud shading method. In this figure, only one portion, including the vertex C of a triangular polygon <b>1</b> comprising vertexes A, B, and C, is illuminated. In this case, lighting processing using diffused light is carried out for the vertex C, but not for vertexes A and B. When the rendering process using the Gouraud shading method with linear interpolation is carried out, the results are then as shown in the lower portion of FIG. <b>13</b>. The area <b>12</b> in the right half of the polygon <b>1</b> has high brightness and reflects the color of the light source, but gradually toward the left side (in a linear manner), brightness becomes low and the color of the light source is not reflected. Brightness becomes low in the area <b>14</b> in the left half.

Such a display can not correctly show the illumination of only a portion at the left side of the polygon <b>1</b> as shown in the upper portion of FIG. <b>13</b>.

FIG. 14 is a drawing to explain further problems with the Gouraud shading method. In this figure, a spotlight, or the like, illuminates an object comprising a plurality of polygons. The edge of the illuminated area <b>10</b> forms a smooth circle. With the Gouraud shading method as shown in FIG. 13, only the linear changes, according to linear interpolation, can be displayed for the portions of the polygons with vertexes located within the area <b>10</b>. Consequently, the edge becomes uneven as shown with the dotted line <b>12</b> in the figure. Also, as shown in FIG. 12, the lighting processing is not carried out for a polygon when the area <b>10</b> overlaps a portion of polygon but none of its vertexes is located within the area <b>10</b>. In such case, also the edge of the area <b>10</b> becomes uneven in FIG. <b>14</b>.

Furthermore, the luminous intensity within the area <b>10</b> is, generally speaking, great at the center and small at the edges. It is difficult to reproduce this distribution of luminous intensity in a realistic manner using linear interpolation in the Gouraud shading method.

Meanwhile, another image processing method, the Phong shading method, has been proposed. In this method, the normal vector is found for each pixel and lighting processing with the normal vector and light source vector is performed for each pixel. However, the Phong shading method requires an enormous number of calculations and is therefore not realistic for game apparatuses and simulators which require real time processing.

SUMMARY OF THE INVENTION

In view of the aforementioned issues, it is an object of the present invention to provide an image processing apparatus and method, able to perform more realistic lighting processing, and a recording medium for storing a program which has a computer execute the procedure.

It is another object of the present invention to provide an image processing apparatus and method, which can perform more realistic lighting processing and minimize increases in calculations while resolving the problems with Gouraud shading, and a recording medium for storing a program which has a computer execute the procedure.

In order to achieve the aforementioned objects, the present invention detects polygons located in a region of diffused light or specular light, divides that polygon and generates subdivided polygons, and for those polygons, carries out lighting processing and rendering processing using linear interpolation according to the conventional Gouraud shading method. In consideration of the efficiency of the image processing, the present invention does not subdivide all the polygons, but rather subdivides the polygons located within the area of diffused light or specular light. Or, the present invention subdivides only those polygons located within areas where the intensity of the diffused light or specular light varies.

With the aforementioned invention, only the parts of the polygons located within areas of diffused light or specular light are subdivided and other polygons are processed at their original size as before. Consequently, linear interpolation with the conventional Gouraud shading method is carried out for the subdivided polygons in areas where changes in brightness and color are to be effectively displayed with lighting processing. More realistic lighting processing can thereby be used to generate images without a corresponding increase in processing.

To attain the aforementioned objects, the present invention is an image processing apparatus, for generating image data from data for a plurality of polygons, comprising: vertex data generating portion for generating polygon data comprising vertex data which includes vertex coordinates, color or brightness data, and light source data comprising data of illuminated area by a light source; dividing those of the aforementioned plurality of polygons which are located within the illuminated area to generate new vertex data; and carrying out processing of reflected light in relation to the light source, for the color or brightness data of the vertexes located within the illuminated area; and a rendering processing portion for generating color data for the pixels within the aforementioned polygons using the aforementioned vertex coordinate, color or brightness data generated above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing to explain an embodiment of the present invention;

FIG. 2 is a drawing for explaining diffused light and specular light;

FIG. 3 is a drawing showing a lighting model;

FIG. 4 is a figure for explaining the method for distinguishing polygons subject to division in the case of a lighting model wherein the luminous intensity distribution has a normal distribution form;

FIG. 5 is a figure for explaining the method for distinguishing polygons subject to division in the case of a lighting model wherein the luminous intensity distribution is in the form of a clamped and cut normal distribution;

FIG. 6 is a figure for explaining the method for distinguishing polygons subject to division in the case of a lighting model wherein the luminous intensity distribution has purely rectangular form;

FIG. 7 is a figure for explaining the method for distinguishing polygons subject to division in the case of a lighting model wherein the luminous intensity distribution has a purely trapezoidal form;

FIG. 8 is a block diagram of the image processing apparatus of an embodiment;

FIG. 9 is a flow chart of the entire process of image processing;

FIG. 10 is a diagram to show the illuminated area processed by Gouraud shading in the event of polygon division in the embodiment;

FIG. 11 is a diagram showing an example of the method for dividing polygons;

FIG. 12 is a diagram to explain problems with Gouraud shading;

FIG. 13 is a diagram to explain problems with Gouraud shading; and

FIG. 14 is a diagram to explain problems with Gouraud shading.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Below, the preferred embodiment of the present invention is explained with reference to the figures. However, these embodiments do not limit the technical scope of the present invention.

FIG. 1 is a drawing to explain an embodiment of the present invention. FIG. 1A corresponds to the situation explained using FIG. 12 of the background art. In this case, an area <b>10</b> of diffused light and specular light is positioned over part of a polygon <b>1</b> comprising vertexes A, B, and C. In order for the area <b>10</b> to be reflected in the polygon <b>1</b>, although the vertexes of the polygon <b>1</b> are not located within the area <b>10</b>, new vertexes D, E, and F are generated and the polygon <b>1</b> comprising vertexes A, B, and C is divided into four parts. As a result, the vertex D becomes located within the area <b>10</b> and lighting processing is carried out for the vertex D. As a result, area <b>24</b> of a polygon with vertexes A, E, and D, area <b>25</b> of a polygon with vertexes D, E, and F, and area <b>26</b> of a polygon with vertexes D, F, and C come to have high brightness and are influenced by the color of the light source.

FIG. 1B corresponds to the situation explained using FIG. 13 of the background art. In FIG. 13, the vertex C, at least, is located within the area <b>10</b>, but it is not possible to correctly represent the edges of the area <b>10</b> by applying linear interpolation, according to a pure Gouraud shading method, to the polygon <b>1</b>. In the embodiment of the present invention, as in FIG. 1B, the polygon is subdivided and new vertexes D, E, and F are generated, even when some of the original vertexes are within the area <b>10</b>. Then vertex data are generated for those vertexes and rendering processing of Gouraud shading method is carried out. As a result, the right half of the area <b>27</b> of the polygon with vertexes D, E, and F is provided high brightness and is influenced by the color of the light source.

Furthermore, the present invention is not limited to the aforementioned example and can distinguish polygons subject to division corresponding to an established lighting model. A polygon subject to division is subdivided by the generation of new vertexes. For example, a polygon, part of which is located within an area where the luminous intensity of the lighting model varies (gradient of luminous intensity), is subject to division. Or, in the case where a lighting model includes an area has a constant luminous intensity and other areas have a luminous intensity of zero, only the portions of a polygon within that area where intensity is constant are subject to division. Polygons located entirely within or outside of the area are not subject to division.

In image processing using computers, the lighting model is determined as an arbitrary and artificial standard by the designer. Consequently, it is impossible to explain how polygons are to be selected for subdivision according to a lighting model. However, if polygons sorted according to a uniform standard are subject to division, it is possible to generate a more realistic, lighting processed image than is the conventional case of applying the Gouraud shading method uniformly to all the original polygons.

FIG. 2 is a figure for explaining diffused light and specular light. FIG. 2 shows a light source <b>20</b> and a polygon <b>1</b>, an area <b>10</b>D of diffused light from the light source <b>20</b>, and an area <b>10</b>S of specular light from a light source <b>22</b>. When the light source <b>20</b> is directional in a single direction, diffused light is generated in an area <b>10</b>D directly illuminated by light from this source. The intensity of this diffused light can be expressed as the inner product, cos θ, of the light vector L and normal vector N. For example, the intensity I of the diffused light can be expressed as follows with intensity I_lof incident light from the light source <b>20</b>, the diffuse reflection coefficient K_d, and angle θ of the light vector L and normal vector N:

<maths><formula-text><i>I=I</i>_a<i>K</i>_a<i>+I</i>_l<i>K</i>_dcos θ</formula-text></maths>

I_ais the environmental luminous intensity and K_ais the reflection coefficient of environmental light.

Meanwhile, FIG. 2 shows a light source <b>22</b> and an area <b>10</b>S of specular light where the light source <b>22</b> is mirror-reflected by the polygon <b>1</b>. This specular light I_scan be expressed as follows with intensity I_lof incident light from the light source <b>22</b>, the reflection coefficient K_s, and angle α of the reflection vector R and sight vector S:

<maths><formula-text><i>I</i>_s<i>=I</i>_l<i>K</i>_scosⁿα</formula-text></maths>

In either of these cases, for a lighting model with light spreading in uniform conic angles φ_d, φ_sfrom the light source, the influence of the reflected light of both must be provided to a polygon located within the areas illuminated by those conic angles φ_dand φ_s. As understood from the foregoing explanation, the effects of the light source must be applied to polygons located within areas illuminated by light from the light source.

FIG. 3 is a drawing showing a lighting model. This lighting model is an example wherein the luminous intensity from the light source <b>29</b> has a normal distribution depending on the conic angle φ. Generally, the central portion <b>31</b> of the conic angle of the light source is the area where the luminous intensity is strong and the peripheral portion <b>32</b> is an area where the luminous intensity is weak.

In image processing using computers, the lighting model sometimes uses an intensity distribution wherein the luminous intensity of the central portion of the luminous intensity distribution is clamped and the luminous intensity outside of the peripheral portion of the light distribution is cut. In other words, this is the luminous intensity distribution <b>41</b> shown with the solid lines in the figure. A distribution of <b>256</b> gradations, for example, is allocated to each luminous intensity; thereby making it possible for the human eye to distinguish finer differences in luminous intensity with the use of few gradations.

As discussed above, the lighting model is determined by a designer's arbitrary standard; there are no limits to the type used. However, the lighting model can generally be expressed with the conic angle φ and the distribution of luminous intensity, as shown in FIG. <b>3</b>. In the present embodiment, polygons affected by diffused reflected light and mirror-reflected light (specular light) are distinguished according to this distribution.

FIG. 4 is a figure for explaining the method to distinguish polygons subject to division in the case of a lighting model where the luminous intensity distribution is in the form of normal distribution. In this example, an area <b>30</b> where the luminous intensity I_lis not zero is defined. When even part of a polygon is located within this area <b>30</b>, that polygon is divided and new vectors generated. For example, in FIG. 4, at least a portion of each of the polygons P<b>1</b>, P<b>2</b>, and P<b>3</b> is located within the area <b>30</b>. These polygons must therefore reflect the effects of the light source and so are identified as being subject to division. Also, the polygon P<b>4</b> which is located entirely outside of the area <b>40</b> is not subject to division. The aforementioned polygons P<b>1</b>, P<b>2</b>, and P<b>3</b> are identified as being subject to division since the change of luminous intensity thereof is not uniform.

The normal vectors, brightness data, color data, and so forth, of the new vertexes are found, as discussed below, for the newly generated vertexes of the polygons identified as subjects of division. Using the normal vectors, lighting processing is carried out for the texture data, brightness data, and color data. Linear interpolation according to the Gouraud shading method, is carried out on the basis of the data for the new vectors in the rendering process which follows.

FIG. 5 is a figure for explaining the method to identify polygons subject to division in the case of a lighting model where the luminous intensity distribution is a clamped and cut normal distribution. As explained with the lighting model in FIG. 3, this method has been used as a method for allocating the luminous intensity distribution most effectively for a finite number of brightness gradations. Specifically, the clamped area where the luminous intensity is very high cannot be distinguished by the human eye; therefore, the luminous intensity distribution <b>41</b> is preferably clamped to a uniform brightness. Also, areas where the normal distribution approaches zero are large areas which cannot be distinguished by the human eye; therefore, the distribution <b>41</b> is preferably cut off below a certain value. As a result, the luminous intensity distribution <b>41</b> in the lighting model in FIG. 5 becomes as shown with the solid lines in the figure.

In this case, the clamped area <b>32</b> and the area <b>31</b> where the luminous intensity is not zero are defined. The polygon P<b>10</b> is located entirely within the clamped area wherein the luminous intensity distribution is uniform; new division is therefore unnecessary. The polygons P<b>11</b> and P<b>12</b>, located within both area <b>32</b> and area <b>31</b>, are identified as polygons subject to division because the luminous intensity varies non-linearly therein. Furthermore, the polygon P<b>13</b>, located between areas <b>31</b> and <b>32</b>, as well as polygons P<b>14</b> and P<b>15</b>, parts of which are within area <b>31</b>, are identified as polygons subject to division because the luminous intensity varies non-linearly therein. The polygon P<b>16</b>, located entirely outside of area <b>31</b>, is not subject to division.

In the example in FIG. 5, it is not necessary to generate new vertexes and subdivide the polygon P<b>10</b>, all vertexes of which are within areas where luminous intensity does not vary, such as area <b>32</b>.

In the example in FIG. 5, the gradient of the luminous intensity distribution is constant for the polygon P<b>13</b>, all vertexes of which are located within the donut-shaped area between area <b>31</b> and area <b>32</b>. Consequently, it is acceptable that this polygon not be subdivided, since the results of linear interpolation in the glow shading method will be the same, even if this polygon P<b>13</b> is subdivided. An algorithm for identifying polygons to be divided can be determined in accord with the display results and the display objective.

FIG. 6 is a drawing to explain the method for identifying polygons subject to division in the case of a lighting model wherein the luminous intensity distribution is a simple rectangle. The model in FIG. 6 concerns a light source having uniform luminous intensity. Specifically, this is a model wherein the entire area within the conic angle from the light source has uniform luminous intensity. The area <b>33</b> is defined for the luminous intensity distribution <b>43</b>.

With the lighting model in FIG. 6, the polygon P<b>20</b>, all vertexes of which are located within the area <b>33</b>, is not subject to division because the luminous intensity therein is constant. Also, polygons such as P<b>21</b> and P<b>22</b>, parts of which are located within area <b>33</b>, are subject to subdivision. These polygons P<b>21</b> and P<b>22</b> include areas illuminated by light and areas not illuminated by light. The sharp edge form of the area <b>33</b> cannot be represented with a rendering process using simply linear interpolation. Consequently, subdividing these polygons makes it possible to display the edge form more sharply. Moreover, the polygon P<b>23</b> located outside the area <b>33</b> is of course not subject to subdivision.

FIG. 7 is a drawing to explain the method for identifying polygons subject to division in the case of a lighting model wherein the luminous intensity distribution is a simple trapezoid. The model in FIG. 7 concerns a lighting model with a luminous intensity distribution <b>44</b>, wherein the normal distribution form <b>40</b> in FIGS. 2 and 4, for example, is clamped and cut, and the inclined portion is made linear. The areas <b>34</b> and <b>35</b> are defined for the luminous intensity distribution <b>44</b>.

As in the aforementioned models, the polygon P<b>30</b>, having all vertexes located within area <b>35</b>, is not subject to subdivision. The polygons P<b>31</b> and <b>32</b>, present in both areas <b>35</b> and <b>34</b>, are subject to subdivision because of the varying changes of luminous intensity therein. As a result, the edge form of the area <b>35</b> can be more sharply represented. Furthermore, the polygon <b>33</b>, all vertexes of which are located in the area between areas <b>35</b> and <b>34</b>, is not subject to subdivision. Because the luminous intensity in this polygon changes linearly, the luminous intensity for the vertex data will also change linearly, even if they are found using linear interpolation. The polygons P<b>34</b> and P<b>35</b>, parts of which are located in area <b>34</b>, are subject to subdivision. The polygon <b>36</b>, outside of area <b>34</b>, is not subject to subdivision.

As above, the determination of whether a polygon is subject to division varies depending on the light distribution model for the light source. However, in linear interpolation from the vertex data in the Gouraud shading method, the standard for a polygon being subject to division can be whether the effects of the light source are represented with sufficient realism. One standard, as in the example explained above, is a polygon wherein the luminous intensity changes non-linearly. In effect, this is a polygon located in an area having variations in the gradient of luminous intensity (changes in luminous intensity). When the luminous intensity within the polygon is constant, the polygon does not need to be divided, of course; and luminous intensity is constant within the polygon even after linear interpolation. Also, in the case where the luminous intensity in the polygon changes constantly (uniform gradient), the polygon does not need to be divided because the luminous intensity changes linearly and at a constant proportion, even upon linear interpolation.

Furthermore, in areas with large changes to the gradient of luminous intensity, it is preferable to increase the extent of subdivision in order to more realistically represent the effects of the light source. Meanwhile, in areas where changes to the gradient of luminous intensity are small, the effects of the light source can be represented with sufficient realism with even a small degree of subdivision. In the example in FIG. 5, the polygons P<b>11</b> and P<b>12</b> may be more finely divided and the polygon P<b>13</b> only roughly divided.

FIG. 8 is a block diagram of the image processing apparatus of the present embodiment. An operation portion <b>58</b> for an operator to carry out input operations, a speaker <b>59</b> for outputting sounds, and a display apparatus <b>66</b> are connected to the image processing apparatus <b>50</b> of the present embodiment.

In the image processing apparatus <b>50</b>, the following are connected with a shared bus <b>55</b>: an I/O processing portion <b>51</b> connected to the operation portion <b>58</b>, ROM <b>53</b> wherein programs or the like for games or simulations are stored, RAM <b>52</b> used in calculation processes, CPU <b>54</b> for executing the aforementioned programs, and sound processing portion <b>56</b>. The sound processing portion <b>56</b> is connected to the sound memory <b>57</b>, which stores sound sources and the like; the sound processing portion <b>56</b> processes sound according to sound data generated by the CPU <b>54</b> and outputs the generated sound signals to a speaker <b>59</b>. The aforementioned program may also be stored in RAM <b>52</b> from outside the apparatus.

The CPU <b>54</b> is related to the polygon vertex data generating portion. In respond to the input signals from the operating portion <b>58</b>, the CPU executes the program within the ROM <b>53</b>, and generates polygon data composing an object. The polygon data usually comprises vertex data such as texture coordinates, transparency, color data, brightness data, normal vectors, and three-dimensional vertex coordinates composing the polygon. Furthermore, the CPU <b>54</b> executes the aforementioned program and generates light source data.

Moreover, the aforementioned color data comprises data for the intensity of the three primary colors red (R), green (G), and blue (B), for example, and includes information on both color and brightness. Meanwhile, the texture data is data relating to the pattern of the polygon and is said to be color data having a standard brightness. Consequently, the aforementioned color data is generated from the brightness data and texture data. As vertex data, the color data and texture data comprise that information itself, but these may also be address and coordinate data pointing to data stored in a separate memory buffer.

The CPU <b>54</b> also executes programs in ROM <b>53</b> and carries out light source calculations, including color calculations for the vertex data with respect to the light source. With the present embodiment, it is determined at the light source calculation stage whether the vertexes of each polygon are located within areas illuminated with the lighting model. It is determined whether to subdivide the polygon according to that positional relationship. For polygons which are subject to division, new vertexes and vertex data are generated and then the same type of light source calculations are made. This process is discussed later in detail.

Vertex data for which the aforementioned lighting processing has been performed are supplied to the geometry processing portion <b>60</b>. The geometry processing portion <b>60</b> performs similarity transformation processing, such as rotating, expanding, or contracting a polygon within three-dimensional coordinates, and perspective conversion processing, such as converting a polygon within the two-dimensional coordinates on the display screen according to the perspective.

Polygon data comprising vertex data within these two-dimensional coordinates are supplied to the rendering processing portion <b>61</b>. The rendering processing portion <b>61</b> uses the vertex data and performs texture generation and color calculations for each pixel according to the raster-scanning method; it finds color data for each pixel and writes the color data for each pixel to the frame buffer <b>64</b>. A texture data buffer <b>62</b>, wherein texture data is stored, and a Z buffer <b>61</b>, wherein the Z value showing the depth of each pixel within the two-dimensional coordinates (usually, a 1/Z value for ease of calculation) is stored, are generally connected to the rendering processing portion <b>61</b>. The texture data is taken from the texture buffer <b>62</b> on the basis of the pixel texture coordinates found with linear interpolation from the texture coordinates of the vertex data. Color data are found from the brightness values and this texture data (color data having standard brightness). For backside processing, the Z values of the pixels during processing are compared with the Z values in the Z buffer <b>63</b>; only the image data of the pixels furthest to the front on the screen are stored in the frame buffer <b>64</b>. The Z values of those pixels are then written to the Z buffer <b>63</b>.

Before the rendering process, light source calculations, to add the influence of the light source, are already carried out for color or brightness data of the vertex data. Polygons in areas illuminated by the light source are then subdivided and light source calculations are performed for the newly generated vertex data. Consequently, the rendering processing portion <b>61</b> performs rendering for pixels by linear interpolation using just the vertex data supplied; the rendering processing portion <b>61</b> is thereby able to generate more realistic images of areas illuminated by light from the light source.

In a conventional image processing apparatus, the processing of the polygons by the aforementioned geometry processing portion <b>60</b> and rendering processing portion <b>61</b> takes the form of pipeline processing, with the polygon data supplied being processed successively and in order. In the case of processing with a general purpose computer, the geometry processing and rendering processing are carried out according to processing programs for each.

Image data, comprising a single frame of color data stored in the frame buffer <b>64</b>, are supplied to the video processing portion <b>65</b> and output as an image signal to the display <b>66</b>.

FIG. 9 is a flow chart for the entire image processing procedure using the aforementioned image processing apparatus. As explained using FIG. 8, the vertex data for the polygons are generated in calculations by the CPU <b>54</b>. The polygon vertex data include three-dimensional coordinate values, normal vectors, color data, brightness data, and texture data, as explained above. The CPU <b>54</b> also generates light source data and viewpoint data (S<b>10</b>).

The light source data for a spotlight, for example, includes light source position coordinates, intensity, color, distance traveled by light from the light source, conic angle Φ as explained using FIG. 2, orientation, and the like. The viewpoint data also includes the location of the viewpoint (three-dimensional coordinates).

In Step S<b>12</b>, the CPU <b>54</b> performs light source calculations for each vertex. Environmental light is the intensity of light applied uniformly, as explained above; the aforementioned I_aK_dis found from the light source data. The reflected light from this environmental light is applied to all vertexes.

The processing for diffused light is performed for the vertexes located within the conic angle φ_dof the lighting model. The luminous intensity at a vertex is found according to the luminous intensity distribution attained from the light source data. The intensity of reflected light due to diffused light (I_lK_dcos θ) is found from the inner product (cos θ) of the light vector and normal vector.

The processing for specular light is also performed for the vertexes located within the conic angle φ_sof the lighting model. The luminous intensity at a vertex is found according to the luminous intensity distribution attained from the light source data. The reflection vector is found on the basis of the normal vector at that vertex; the intensity of reflected light due to specular light (I_s=I_lK_scosⁿα) is found from the power of the inner product of the reflection vector and sight vector.

Using each of the aforementioned reflected luminous intensities, the color data of the vertex is corrected by multiplying the light source color data with the original color data of the vertex. The brightness data is also corrected with a similar calculation. Consequently, the color data and brightness data for the vertexes located within an area illuminated by the light source include the effects of the light source.

In the present embodiment, the vertex data for a polygon located within an area illuminated by the light source is used in linear interpolation to generate an image thereof. When it is not possible to reproduce the illuminated area or generate an image with a sufficient sense of place, that polygon is subdivided. To that end, the coordinate data in the vertex data is used to determine whether each vertex is located within the conic angle of the lighting model. On the basis of this determination, the algorithm explained using FIGS. 4-7 is then used to determine whether the polygon subject to division (S<b>16</b>).

When a polygon is found to be subject to division, that polygon is divided and new vertexes generated (S<b>18</b>). The generation of the new vertexes can be carried out by finding, through linear interpolation, vertex data using a method of dividing geometric intervals. The degree of subdivision is as explained using FIGS. 4-7. For example, a polygon located in an area where the gradient of luminous intensity distribution varies greatly is preferably more finely subdivided. A polygon located in an area where the gradient of luminous intensity varies little does not need to be very finely subdivided. The new vertex data undergo lighting processing in a repetition of Step S<b>12</b>. As a result, diffused light and specular light are added when the newly generated vertex is located in an illuminated area. New vertexes are generated when the polygon is divided and require the increase of the aforementioned light source calculations and subsequent rendering processing. The degree of subdivision is therefore determined according to the trade off between the quality of the image produced and processing speed.

In the present embodiment, the process of dividing one polygon is carried out once. Consequently, the polygons generated as a result of this division are not further subdivided. In Step S<b>14</b>, the vertex data after subdivision undergoes similarity transformation and perspective conversion (S<b>20</b>), without undergoing the process (S<b>16</b>) for determining whether division is necessary. As a result, the coordinate data of the vertexes is transformed to two-dimensional coordinate data for the display screen. This process S<b>20</b> is carried out with the geometry processing portion <b>60</b>.

As a result of the perspective conversion process, the size of the polygon on the screen where it is actually displayed is confirmed. Consequently, the degree of subdivision of the polygon can be determined at this time. In that case, the vertex data generated in the repeated Step S<b>12</b> must be processed. When the image processing apparatus has the limits of its capacity, the polygon is divided before the perspective conversion, as explained above.

The vertex data which has undergone perspective conversion is then supplied to the rendering processing portion <b>61</b> and undergoes rendering processing, as explained above, using the texture buffer and Z buffer (S<b>22</b>). Image data for each pixel generated thereby is stored in the frame memory <b>64</b>. The aforementioned Steps S<b>12</b>-S<b>22</b> are performed for all pixels in a single frame. The image signal is finally sent to the display <b>66</b> by the video processing portion <b>65</b> and displayed (S<b>24</b>).

FIG. 10 is a drawing to show an illuminated area processed by Gouraud shading when a polygon is divided as in the present embodiment. This corresponds to FIG. 14 of the background art. In FIG. 14, the dotted line <b>12</b> showing the edge of the illuminated area has a form different from the actual circle. In the example in FIG. 10, however, the form is more circular. The example in FIG. 10 shows the application of the present embodiment to the case of the lighting model in FIG. <b>6</b>. Specifically, only a polygon located across the edge of the illuminated area <b>10</b> is subject to division. The polygons along the edge portion are accordingly subdivided. As a result of the subsequent linear interpolation for the Gouraud shading process, the edge of the illuminated area displayed becomes as shown with the dotted line <b>14</b>. As the degree of subdivision increases, the form can be made more circular.

FIG. 11 is a drawing to show an example of the method for dividing polygons. In this example, from among a set of triangular polygons of varying sizes, those polygons located on the edge of an area <b>10</b> illuminated by the light source are divided into four equal parts. As explained above, the number of divisions is determined according to the amount of change in the gradient of luminous intensity distribution of the lighting model. In the example in FIG. 11, the number of divisions is equal regardless of the sizes of the polygons. The reason why the number of divisions is equal in this example is as follows. As explained for perspective conversion processing, it is ideal to determine the number of divisions according to the amount of change in the gradient of the luminous intensity distribution of the aforementioned lighting model with considering the sizes of the polygons which are ultimately displayed. In the present embodiment explained above, however, the pertinent polygons are divided before perspective conversion in order to reduce the processing load. Consequently, in the example in FIG. 11, the processing can be further reduced or simplified by dividing all polygons subject to division into an equal number of divisions.

As explained above, even when performing rendering processing according to the Gouraud shading method, the present invention can generate an image which more realistically shows the effects of light by dividing only those polygons located in areas illuminated by the light source. Moreover, the overall processing load is not greatly increased because a simple process is used to divide only some of the polygons.

Consequently, the present invention can be applied to an image processing apparatus in a game apparatus or the like for which costs must be reduced. An image which underwent more realistic lighting processing can be generated while increased costs are suppressed.