Method and apparatus for object detection and background removal

CA 02249140 2001-06-05

METHOD AND APPARATUS FOR OBJECT DETECTION

AND BACKGROUND REMOVAL

BACKGROUND OF THE INVENTION

The present invention relates to a method and apparatus for object detection and background removal, and storage media for storing therein a program for the sake of obtaining from an object an image with a background removed. More particularly, this invention relates to a method and apparatus for object detection and background removal, and storage media for storing a program which can obtain from an object an image having a virtually constant background.

Description of the Related Art

A method and apparatus for object detection and background removal, and a storage media for storing a program thereof, is applicable to various fields of application. The object detection and background removal method is based on technology which can isolate an object from an image with a virtually constant background. The object is photographed with the exception of a background area. Figure 1 is a view showing one example of a processing target image. In the input image signal 1 of Figure 1, a target object 3 whose photograph is taken is shown having a virtually constant background area 2. If it is capable of being isolated completely, only the target object 3 will remain, while removing the background area 2. It is also capable of being applied to image composition techniques in the field of computer vision instead of object recognition processing, robot visual sensation, or chromakey techniques.

In a television program, a scene in which a person is associated with another background is often televised. The chromakey technique, which is one of techniques for cutting a target object out of the background, is frequently used by television stations. The chromakey technique is described in accordance with the

CA 02249140 2001-06-05 literature: Television Image Rejection Engineering Handbook (Television Society, pp. 704, 1990).

One of the techniques for isolating the target object to be removed from the background is described in accordance with the literature: Japanese Patent Application Laid-Open No. HEI 2-206885 Image Processing Device. In the present literature, by way of processing the target image, the image is to be composed of a background having intermediate brightness, and the object which comes to be the target of cutting consists of either an area brighter than the background or a darker area than the background. The threshold which is established by some way or other is utilised, subsequently, implementing threshold processing for isolating the pixel with a brightness value beyond the threshold, thus isolates the pixel which is brighter than the background. Similarly, by virtue of threshold processing and utilising a separate threshold, the pixel darker than the background can be isolated. Thus, it is capable of isolating the whole object by synthesizing the results of the two threshold processing.

There is described one of the techniques for calculating stably the characteristic quantity such as the center of gravity location or the area thereof from the target object isolated out of the background. It is described in accordance with the literature: the Japanese Patent Publication No. HEI 4-116778 Image Processing

Method.

In general, there mostly exists a so called shading, which is a gentle slope of the brightness value in the background. In most cases, the brightness value of the object also has various values, and fluctuates. In these cases, it is difficult to detect a boundary line between the background and the object accurately over the whole area by the use of one kind of threshold. Accordingly, in the present literature, the user establishes two kinds of thresholds. One threshold is capable of detecting the target object accurately and another threshold is capable of detecting the background

CA 02249140 2001-10-23 area accurately. The user sets these two kinds of thresholds interactively to be binarized, and in terms of the pixel having an intermediate value in between the two thresholds, there is allocated a value corresponding to the intermediate value. Due to the use of this method, even though the boundary line is incapable of being obtained accurately, it is capable of obtaining the center of gravity location of the target object stably.

Another binarization method is described referring to the literature Image Analysis Handbook (Supervision ofM.Takagi Y. Shimoda, Tokyo University Publication Party, pp. 502-505, 1991).

In the method introduced thus far, the user determines the threshold applied to the whole area of the picture in such a way that the user sets the threshold interactively while watching the image of the processing result. As shown in the literature, there is proposed a p-tile way, a method of Otu, or a method of Kittler by way of an automatic determining method of the threshold. For instance, the method of Otu is referred to as a discriminant analysis way in that on the assumption that a gray level histogram of an image is constituted by the sum of two normal distributions, and this is the method for obtaining the threshold enabling them to be separated completely. At the same time, it is capable of calculating a degree of separation which is in use for measuring a scale that is capable of judging to what degree the two distributions are separated, and is capable of judging whether or not bimodality of the histogram is high. Namely the degree of separation capable of being utilized is determined by use of a determination scale, and then determining whether or not the threshold being obtained is appropriate.

However, as described above, generally, the shading exists mostly in the background, and accordingly there is a limit in the binarization method which sets the same threshold over the whole picture. Now, there is a method for a dynamic threshold

CA 02249140 2001-06-05 processing for calculating the most suitable threshold in every pixel. According to the literature, dynamic threshold processing is classified into two methods: a movement mean method and a sectional image division method.

The movement mean method is a simple method, in that when it causes the brightness value of some pixels to be binarized, obtaining the mean value of the sectional image including the neighbourhood thereof is to be taken as the threshold.

The sectional image division method determines automatically the most suitable threshold in every respective sectional image while dividing the whole picture into a plurality of sectional images. The method causes the determined thresholds to be connected smoothly, thus constituting surface of the threshold over the whole picture so that the image is binarized.

There is described one example of the sectional image division method referring to the literature. Firstly, the image is divided into small area sectional images. Figure 2 is a typical view. When the input image signal 1 shown in Figure 1 is inputted, dividing the inputted image into sectional image signals 4, 5, 6, 7, 8 and so forth such that these sectional image signals overlap one another, as shown in Figure 2. Within the respective sectional image signals, the threshold and the degree of separation are calculated at this position while applying the binarization method of Otu. In the sectional image whose degree of separation comes into high rather than a value set beforehand, since there is included both the background and the target object, thus being judged that appropriate threshold is obtained. The appropriate threshold is adopted by way of the threshold value in the pixel of the center position of the sectional image, while in another position of the pixel, the thresholds are connected smoothly, and thus the surface area of the threshold of the whole picture is determined. By virtue of the above described procedure, it is capable of obtaining an appropriate threshold over the whole

CA 02249140 2001-06-05 picture, thus enabling a suitable binarization result to be obtained even though there exists shading or the like.

There are two problems with the chromakey technique. The background constituted by a peculiar colour beforehand is incapable of being used with a target object whose colour is identical with the background colour. The method of the literature: Japanese Patent Application Laid-Open No. HEI 2-206885 is incapable of being applied to a case where the target object has a brightness value which bears a close resemblance to the background, because the target object should be photographed by way of which the brightness value of the target object is always brighter than the background or the brightness value of the target object is darker than the background. The method ofthe literature: Japanese Patent Publication No. HEI 4-116778 is not intended to obtain a boundary in between the background and the target object accurately from the beginning, and accordingly, it is incapable of being utilised in the case where an accurate boundary line is required.

With respect to the method which applies one threshold to the whole picture, it is difficult to obtain a satisfactory boundary line in most cases, if shading exists in the background, or in cases where the brightness value of the object or colour of the object is not one of simplicity. This is referred to in the literature: Japanese Patent Publication No. HEI 4-116778, or in the literature: Pixel Position Analysis

Handbook.

The objective of the movement mean method or the sectional image division method of dynamic threshold processing is to be applied to such a case. In the movement mean method or the sectional image division method of dynamic threshold processing, it is necessary to set a presupposition that the background and the object which are constituted by one kind of brightness value or colour in the position of the pixel should be photographed. For this reason, it is incapable of applying the method

CA 02249140 2001-06-05 to the case where the target object is constituted by a plurality of brightness values or colours. For instance, in the politics section or the sports section of the newspaper, a photograph of a large number of persons faces appears therein while making them even so as to be in the same position of faces and size with the identical background. When such a space is to be edited, it is necessary to remove the original background precisely with regard to contours of images because the origin of the photograph of the face is photographed with respective different backgrounds or different sizes. In most cases the background is constant, however it is incapable of being estimated beforehand because if shading exists, the colour of the hair or the skin, or clothes of the people in the photograph are not constant. In particular, it is difficult to separate between a whitish background and a long-sleeved sport shirt. Accordingly, it is extremely difficult to detect automatically the contours of a person's image accurately by the conventional method. Under such conditions, a man of experience implements the work for removing the background by hand.

There is one object which has a background in which shading exists uniformly, and which consists of a single or a plurality of brightness or colour. There is another object which has brightness and colour nearly equal to that of the background. Namely, there is the problem that it is extremely difficult to detect such objects precisely with regard to contours in relation to someone else.

SUMMARY OF THE INVENTION

In view of the foregoing, it is an object of the present invention to provide a method and apparatus for object detection and background removal, and a storage medium for storing a program which enables an object to be automatically detected and to isolate precisely and accurately the contours of the object.

CA 02249140 2001-10-23

According to a first aspect of the present invention, for achieving the above-mentioned object, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, comprising the steps of using an image, consisting both of a background and an object of detection target, to be an input image by way of input processing of the image, and selecting a sectional image including only the background while dividing the input image into a sectional image by way of a background only sectional image selection process, estimating a background on the input image based on the sectional image including the background by way of a background estimation process, and comparing the estimated background with the input image by way of a comparison process, whereby the object of detection target is isolated from the input image.

In the first aspect of the present invention, there is provided a method for object detection and background removal such that a location of a pixel, which is constituted by a background and an object of detection target, is inputted, and subsequently, a sectional image including only a background is selected while dividing the inputted location of the pixel into sectional images, then, estimating a background on the input image based on a sectional image including the background concerned, before comparing the estimated background with the above input image, thus isolating only the object of detection target.

Namely, in the first aspect of the present invention, a method for object detection and background removal firstly selects a sectional image including only a background while dividing the image into sectional images, before estimating backgrounds in all of the images based on the sectional image concerned, thus enabling the background and unknown object for example, an object with brightness

CA 02249140 2001-10-23 colour distribution, to be separated to be detected accurately by an estimation of the background in all of the images based on the sectional image concerned.

In general, in order to separate accurately the background and an object of detection target, it is necessary to find as precisely as possible the distribution of a characteristic value such as brightness, colour, edge and so forth which appear on the image by way of the background or an object. In the first aspect of the present invention, a sectional image including only a background is selected, before the background in the whole picture is estimated, by using a technique of interpolation and extrapolation from the sectional image.

According to a second aspect of the present invention, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately comprising the steps of, taking an image consisting both of a virtually constant background and an object of detection target, to be an input image by way of input processing of the image, and calculating a statistic in each respective sectional image while dividing the input image into sectional images by way of a statistic calculation process, selecting a sectional image including only the background based on the statistic calculated in the statistic calculation process by way of a background only sectional image selection process, estimating a statistic of the whole picture area from a statistic of a sectional image including only the background by way of a statistic estimation process, determining a threshold in the whole picture area from the estimated statistic by way of a threshold determination process, and comparing the threshold determined in the whole picture with the input image by way of a comparison process, whereby the object of detection target is isolated from the input image.

In the second aspect of the present invention, an object detection and background removal method calculates a statistic in each of the respective sectional

CA 02249140 2001-10-23 images while dividing the input image into sectional images, with an image constituted by a virtually constant background and an object of detection target as inputs, and subsequently determining a threshold in the whole picture from the statistic calculated previously, so that it causes a detection target to be isolated while comparing the determined threshold in the whole picture with the above input image.

According to a third aspect of the present invention, there is provided a method of object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein said statistic calculation process consists of a sectional image division process dividing the input image into sectional images and a mean value and a standard deviation calculation process calculating a mean value and a standard deviation of a prescribed characteristic value of the divided sectional image.

In the second aspect of the invention, the object detection and background removal method calculates a statistic in each respective sectional image while dividing the input image into sectional images, as in the process of the second aspect, and in addition thereto, for instance, the object detection and background removal method of the third aspect causes a mean value and a standard deviation of the brightness of a sectional image to be calculated while dividing the input image into sectional images.

Namely, in the third aspect of the present invention, an object detection background removal method divides the input image into sectional images, and subsequently, it is a characteristic of the method to calculate a mean value and a standard deviation using characteristic information of pixels within the respective sectional images. As shown in Figure 1, when an input image signal 1 which i constituted from a background area 2 and a target object 3 is inputted to an apparatus of the invention of the third aspect, firstly, the input image signal 1 is divided into sectional

CA 02249140 2001-06-05 images 1-A, 1-B, ···, 9-I as shown in Figure 4. Then, it causes a mean value and a standard deviation of brightness and so forth to be calculated in a location of the sectional image in each respective sectional image signal. For instance, in Figure 4, an area of sectional image 1-A is mentioned below as mark (1) with the number of the row and column arranged.

Cl-A (1)

A brightness value for instance, a spot of co-ordinates (x,y) is mentioned as following (2).

Ix,y (2)

At this time, for instance, a mean value of the brightness in the sectional image i-p is defined with equation (3). Here, a denominator of the equation (3) is an area of the sectional image.

Mi-p - { Σ x,yeCi~pIx,y) / ( Σ K,yeCi-pl) (3)

A standard deviation is defined with an equation (4) utilizing the mean value.

O*i-p = V { Σ x,_yeCi-p ( Ix,y“ JX i-p) ²} / {Σχ,γ€θί-ρΐ} (4)

The mean value and the standard deviation are calculated over the whole sectional images based on the equations (3), and (4).

CA 02249140 2001-10-23

Further, in the third aspect of the present invention, a geometrical mean or a harmonic mean or median or the like which are described in the literature: Modern Mathematical Science Dictionary published by Maruzen Co., Ltd, 1991 pp. 495 are capable of being used instead of an arithmetical mean shown in the above equation (3). Similarly, a statistic of an absolute deviation or a quarter deviation or the like are capable of being used instead of the standard deviation shown in the equation (4) or a distribution represented by square thereof.

Furthermore, as shown in Figure 4, the sectional images are arranged in a tile-shaped configuration, however as shown in Figure 2, the sectional images are superimposed with each other in part thereof, or the sectional images are arranged while leaving a space therebetween. All of these configurations are effective by way of the third aspect of the present invention.

According to a fourth aspect of the present invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the sectional image selection process determines the sectional image whose standard deviation of the prescribed characteristic value is of the smallest value as the sectional image whose probability of including only a background is high, and determines the sectional image whose difference of comparison is less than the prescribed threshold to be the sectional image including only the background while comparing a standard deviation of the prescribed characteristic value in another sectional image with the sectional image whose probability of including only the background is high.

In the third aspect of the invention, the object detection and background removal method selects the sectional image including only the background based on the calculated statistic, as in the process of the third aspect, and in addition thereto, the object detection and background removal method of the fourth aspect determines a

CA 02249140 2001-10-23 sectional image whose standard deviation of a characteristic value such as brightness and so forth is of the smallest value, to be the sectional image whose probability of including only a background is high, and subsequently, comparing a standard deviation of the brightness in another sectional image with a standard deviation of the sectional image whose probability of including the background is high, whereby the sectional image whose difference of the standard deviations therebetween is less than the threshold is determined to be the sectional image including only a background.

Namely, in the fourth aspect, the method causes a sectional image including only a background to be selected from sectional images, for instance, selecting a sectional image whose standard deviation of the brightness is of the smallest value, and subsequently, selecting sectional images whose standard deviations bear resemblance to those sectional images selected previously to be sectional images including only backgrounds. On the images inputted thereto, since the backgrounds have virtually even distribution, it is to be expected that a sectional image including only a background has a small standard deviation. For this reason, a sectional image whose probability of including only the background is obtained due to a condition that a standard deviation is of the smallest value. For instance, in Figure 4, the standard deviation of the sectional image 1-A is of the smallest value, thus the sectional image 1-A is judged to be a sectional image whose probability of including only the background is high. The standard deviation of the sectional image 1-A is set to a_bg. There is implemented a determination as to whether or not another sectional image i-p includes only a background in such a way that it is judged due to a threshold processing of an equation (5) using a standard deviation of the sectional image i-p, the standard deviation of the sectional image 1-A, and two constants η.

cr_bg σ>-? η σ + συ (5)

CA 02249140 2001-06-05

Here, 7? V— is a constant given beforehand which is more than 0 and until 1, further,73 o'+ is a constant value given beforehand which is more than 1.

For instance, with respect to a mean value of the brightness, when shading is given on a background, even though the sectional image includes only a background, the mean value of the brightness is of differing values according to the location on the image, thus the mean value of the brightness does not qualify for judgement as to whether or not the sectional image includes only a background. The equation (5) is described by way of one example of a threshold processing, however, for instance, an equation (6), by using constant δ to be more than 0 and so forth, is capable of constituting an effective invention in conformity with a target by defining variously the terms of the threshold processing.

<Xbg — Ô (Ti-p CTbg + Ô G+ (6)

Figure 5 is an example of the selection result of a sectional image including only the background while giving light hatching, thus showing simultaneously an object of detection target 3. A group of sectional images which are images with the exception of the object of detection target 3, and which are not influenced by noise largely, are selected to be sectional images including only the background.

According to a fifth aspect of the present invention, there is provided a method of object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the sectional image selection process selects sectional images in such a way that it causes a specified number of the sectional images in order of the smaller number of a standard deviation of the prescribed characteristic value to be selected, thus taking such

CA 02249140 2001-06-05 sectional images to be the sectional image whose probability of including only the background is high.

In the fourth aspect of the present invention, the object detection and background removal method selects the sectional image whose probability of including only a background is high, as in the process of the fourth aspect, and in addition thereto, the object detection and background removal method of the fifth aspect selects sectional images in specified numbers, in the order of smaller value of a standard deviation of the brightness to be a sectional image whose probability of including only a background is high.

According to a sixth aspect of the present invention, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection process selects sectional images in such a way that it causes the sectional image having the nearest most value to the standard deviation of the prescribed characteristic value to be selected as the sectional image whose prescribed characteristic value of including only the background is highest.

In the fourth aspect of the invention, the object detection and background removal method selects a sectional image whose probability of including only a background is high, as in the process of the fourth aspect, and in addition thereto, the object detection and background removal method of the sixth aspect selects a sectional image whose standard deviation is of the nearest most value of the standard deviation of the brightness to be the sectional image whose probability of including only a background is highest.

According to a seventh aspect of the invention, there is provided a method of object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline, wherein the sectional image

CA 02249140 2001-06-05 selection process selects a specified number of sectional images in such a way that it causes the sectional image, in order, having a nearer number to the standard deviation of the prescribed characteristic value to be selected as the sectional image whose prescribed characteristic value of including only the background is high.

In the fourth aspect of the invention, the object detection and background removal method selects a sectional image whose probability of including only a background is high, as in the process of the fourth aspect, and in addition thereto, the object detection and background removal method of the seventh aspect selects sectional images in answer to the specified numbers in order of nearer value of standard deviation of the brightness to be the sectional image whose probability of including only a background is high.

According to an eighth aspect of the invention, there is provided a method of object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection process selects sectional images in such a way that it causes the sectional image having the nearest most value both to the mean value and the standard deviation of the prescribed characteristic value to be selected as the sectional image whose probability of including only the background is high.

In the fourth aspect of the invention, the object detection and background removal method selects a sectional image whose probability of including only a background is high, as in the process of the fourth aspect, and in addition thereto, the object detection and background removal method of the eighth aspect selects a sectional image whose mean value and standard deviation are the nearest most value of the mean value and the standard deviation of the brightness to be the sectional image whose probability of including only a background is high.

CA 02249140 2001-06-05

Namely, the eighth aspect of the invention specifies for instance, not only the standard deviation of the brightness but also a mean value thereof beforehand when selecting the sectional image whose probability of including oniy a background is high, in comparison with the seventh aspect. If shading is given on a background, it is difficult to utilize the mean value of the brightness and so forth, as when selecting a sectional image including only a background. However, when there is selected a sectional image whose probability of including only the background becomes a criterion of the selection, as when selecting a sectional image including only a typical background, the eighth aspect comes to be an effective invention in conformity with a target.

According to a ninth aspect of the invention, there is provided a method of object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection process selects sectional images in such a way that it causes the sectional images of the number specified having a nearer value to the mean value and the standard deviation of the prescribed characteristic value to be selected in order as the sectional images having a probability of including only the background.

In the fourth aspect of the invention, the object detection and background removal method selects the sectional image having a probability of including a background, as in the process of the fourth aspect, and in addition thereto, the object detection and background removal method of the ninth aspect selects a specified number of sectional images, in order, having a nearer value both to the mean value and the standard deviation of the brightness to be the sectional image having a probability of including a background.

According to a tenth aspect of the invention, there is provided a method of object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection

CA 02249140 2001-10-23 process causes the sectional image whose probability of including only the background is high to be either single image or a plurality of images.

In the fourth aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including only the background is high, as in the process of the fourth aspect, and in addition thereto the object detection and background removal method of the tenth aspect judges a partial image having a high probability of including only a background to be either single sectional image or a plurality of sectional images.

Namely, in the invention of the tenth aspect, there is a partial image having a high probability of including only a background. For instance, as shown in Figure 1, when an image is photographed while controlling so as to locate an object in the center of the image, in this case, it is certain that four corners of the image of the sectional images 1 -A, 9-A, 1-1, 9-1 in Figure 4 are backgrounds. In another case, such as where a photograph of a person appears in a newspaper, both corners located above sectional images 1-A, and 9-A are backgrounds, as noted in Figure 4. As described above, the tenth aspect enables other sectional images including only the background to be selected based on the sectional images which surely include only a background.

According to an eleventh aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline, wherein the sectional image selection process selects a sectional image having a high probability of including only the background from sectional images included in an area.

In the fourth to ninth aspects of the invention, the object detection and background removal method selects the sectional image having a high probability of including only the background, as in the process of the fourth aspect to the ninth aspect, and in addition thereto, when the object detection and background removal method of

CA 02249140 2001-06-05 the eleventh aspect selects a sectional image having a high probability of including only a background, it is selected from a predicted area including only the background instead of all of the sectional images.

According to a twelfth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline, wherein the sectional image selection process selects a sectional image having a high probability of including only the background from sectional images included in a plurality of areas.

In the fourth aspect to the ninth aspect of the invention, the object detection and background removal method selects the sectional image having a high probability of including only the background, as in the process of the fourth aspect to ninth aspect, and in addition thereto the object detection and background removal method of the twelfth aspect selects a sectional image having a high probability of including only a background, from sectional images involved in a plurality of areas.

According to a thirteenth aspect of the invention, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein said statistic calculation process consists of a sectional image division process dividing the input image into sectional images, and a mean value and standard deviation calculation process calculating a skewness from a mean value and standard deviation of the prescribed characteristic value of a sectional image.

In the second aspect of the invention, the object detection and background removal method calculates the statistic in each respective sectional image while dividing the input image into sectional images, as in the process of the second aspect, and in addition thereto, the object detection and background removal method of the thirteenth

CA 02249140 2001-10-23 aspect, for instance, calculates a mean value, a standard deviation, and a skewness of the brightness of a sectional image while dividing the input image into sectional images.

Namely, the invention of the thirteenth aspect newly calculates the skewness in comparison with the third aspect. The skewness of the sectional image i-p is defined based on the literature: “Mathematical Statistics written by Takashi Takeuchi, published by Toyo Keizai, 1963, pp.29 by a following equation (7).

ζί-ρ= (1/ σΐ-ρ³) { Σ x,yeCi-p (I_X,y— Ji i-p) 7 Σ x,_yeCi-_Pl} (7)

As described in the same literature “Mathematical Statistics pp.29, a skewness is used for representing slippage from normal distribution, and also is the standard judging lateral symmetric property of distribution. On the assumption that a background has a virtually constant state in a sectional image, it is capable of being understood thatfor instance, brightness distribution is like a normal distribution, thereby, the skewness is useful in judging whether or not an image is a background. The skewness becomes zero when the distribution agrees with the normal distribution, and the skewness takes a separated value from zero when the distribution gets out of the normal distribution.

In the thirteenth aspect of the invention, a kurtosis which is described in the same literature is capable of being used instead of the skewness. The kurtosis is available to judge whether or not an image is a background, and is an available standard of simplicity in judging the brightness distribution.

According to a fourteenth aspect of the invention, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection process determines a sectional image whose standard deviation of the

CA 02249140 2001-10-23 prescribed characteristic value is the most smallest one, as a sectional image whose probability of including only the background is high, from among those sectional images whose absolute value of skewness of the prescribed characteristic value is less than a threshold, while in other sectional images, the sectional image selection process determines a sectional image having an absolute value of the skewness of the prescribed characteristic value less than the threshold and wherein the difference of a standard deviation of the prescribed characteristic value between the sectional image having a high probability of including only the background and the sectional image concerned is less than the threshold, as the sectional image including only the background.

In the thirteenth aspect of the invention, the object detection and background removal method selects the sectional image including only the background based on the calculated statistics, as in the above process of the thirteenth aspect, and in addition thereto, the object detection and background removal method of the fourteenth aspect determines a sectional image whose standard deviation of the brightness is of the smallest value to be the sectional image having a high probability of including only the background from among sectional images whose absolute value of the skewness of the brightness and so forth is less than the threshold, while in other sectional images, for instance, the fourteenth aspect determines a sectional image having an absolute value of the skewness of the brightness and so forth less than the threshold, wherein the difference of standard deviation between the sectional image concerned and the sectional image having a high probability of including only the background is less than the threshold, to be the sectional image including only a background.

In the invention of the fourteenth aspect, the skewness is utilized when determining whether or not a sectional image includes only the background. For instance,

CA 02249140 2001-10-23 threshold processing can determine whether or not the sectional image includes a background by an equation (8).

I ζΐ-Ρ I <7) ζ (8)

Here, 77 £ is a constant determined beforehand which takes a larger value more than zero (0). According to a determination of the present skewness, it enables a distribution, perhaps for a background, whose brightness value distribution is virtually constant to be investigated within sectional images.

According to a fifteenth aspect ofthe invention, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection process determines a sectional image having an absolute value of a skewness ofthe prescribed characteristic value less than the threshold as the sectional image having a high probability of including only the background.

In the fourteenth aspect of the invention, the object detection and background removal method selects the sectional image having a high probability of including only the background, as in the process ofthe fourteenth aspect concerned, and in addition thereto, the object detection and background removal method ofthe fifteenth aspect, for instance, determines a sectional image having an absolute value ofthe skewness ofthe brightness less than the threshold to be a sectional image having a high probability of including only a background.

Namely, the invention of the fifteenth aspect investigates a distribution to be a background whose skewness is of a small value and whose brightness value distribution is approximately uniform in the sectional images only.

CA 02249140 2001-10-23

According to a sixteenth aspect of the invention, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection process selects a specified number of sectional images, in order, having a smaller standard deviation from among those sectional images whose absolute value of a skewness of the prescribed characteristic value is less than the threshold, and subsequently determining the sectional image concerned as the sectional image having a high probability of including only the background.

In the fourteenth aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including only the background is high, as in the process of the fourteenth aspect, and in addition thereto, the object detection and background removal method of the sixteenth aspect, for instance, selects a specified number of sectional images, in order, having a smaller value of the standard deviation from among those sectional images whose absolute value of the skewness of the brightness and so forth is less than the threshold, to be the sectional image whose probability of including only a background is high.

According to a seventeenth aspect of the invention, there is provided a method for object detection and background removal for enabling the outline of an object to be automatically detected minutely and accurately, wherein the sectional image selection process selects a specified number of sectional images, in order, having a smaller standard deviation from among those sectional images whose absolute value of a skewness of the prescribed characteristic value is less than the threshold, and subsequently determining the sectional image concerned as a partial image whose probability of including only the background is high.

In the fourteenth aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including

CA 02249140 2001-10-23 only the background is high, as in the process of the fourteenth aspect, and in addition thereto, the object detection and background removal method of the seventeenth aspect, for instance, selects a sectional image having a standard deviation of the most nearest value to the standard deviation of the brightness from among those sectional images whose absolute value of the skewness of the brightness and so forth is less than the threshold, thus determining it to be the sectional image whose probability of including only a background is high.

According to an eighteenth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the sectional image selection process selects as many sectional images as specified in the order of having the most nearest value to the standard deviation of the skewness of the prescribed characteristic value from among those sectional images whose absolute value of skewness of the prescribed characteristic value is less than the threshold, and determines the sectional image concerned as the sectional image whose probability of including only the background is high.

In the fourteenth aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including only the background is high, as in the process of the fourteenth aspect, and in addition thereto, the object detection and background removal method of the eighteenth aspect, for instance, selects a specified number of sectional images, in the order of having a nearer value to the standard deviation of the brightness from among those sectional images whose absolute value of the skewness of the brightness and so forth is less than the threshold, thus determined to be sectional images whose probability of including only the background is high.

CA 02249140 2001-10-23

According to a nineteenth aspect of the invention, there is a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the sectional image selection process selects a partial image whose standard deviation is of the nearest most value to the standard deviation of the prescribed characteristic value, from among those sectional images whose absolute value of the skewness of the prescribed characteristic value is less than the threshold, thus determining the partial image concerned as the sectional image whose probability of including only the background is high.

In the fourteenth aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including only the background is high, as in the process of the fourteenth aspect, and in addition thereto, the object detection and background removal method of the nineteenth aspect, for instance, selects a specified number of sectional images, in order, having a nearer value to the mean value and the standard deviation of the brightness from among those sectional images whose absolute value of the skewness of the brightness and so forth is less than the threshold, and thus determining them to be sectional images whose probability of including only the background is high.

According to a twentieth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the sectional image selection process selects as many partial images as specified, in the order of having the nearest most value to the mean value and the standard deviation of the prescribed characteristic value from among those sectional images having an absolute value of the skewness of the prescribed characteristic value less than the

CA 02249140 2001-10-23 threshold, thus determining the partial images concerned as the sectional images whose probability of including only the background is high.

In the fourteenth aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including only the background is high, as in the process of the fourteenth aspect, and in addition thereto, the object detection and background removal method of the twentieth aspect, for instance, selects a specified number of sectional images, in order having a nearer value to the standard deviation of the brightness and so forth from among those sectional images having an absolute value of the skewness of the brightness less than the threshold, thus determining them to be sectional images whose probability of including only the background is high.

According to a twenty first aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the sectional image selection process, where there is given beforehand a probability of including only the background in each sectional image involved in a plurality of areas, determines the sectional image whose probability of including only the background is the highest as the sectional image whose probability of including only the background is high from among those sectional images whose absolute value of the skewness of the prescribed characteristic value is less than the threshold.

In the fourteenth aspect of the invention the object detection and background removal method thereof selects the sectional image whose probability of including only the background is high, as in the process of the fourteenth aspect, and in addition thereto, the object detection and background removal method of the twenty first aspect, wherein the probability of including only the background is given beforehand in every sectional image involved in the area, thus determines a partial image whose

CA 02249140 2001-10-23 probability of including only a background is high to be the sectional image whose probability of including only the background is high from among those sectional images whose absolute value of the skewness of the brightness is less than the threshold.

Namely, the invention of the twenty first aspect selects the sectional image whose probability of including only the background is of the highest value from among those sectional images which satisfy the condition that the absolute value of the skewness is less than the threshold. For instance, as shown in Figure 1, where it is intended to locate an object to the center position, the probability of including only a background in the corners of the image is higher rather than that of the center position. There is given a probability of including only the background with regard to respective sectional images beforehand. For instance, in Figure 6, there is established beforehand the probability of including only the background of the sectional images 1-A, 2-A, 1-B, and 2-B to 0.8, 0.6, 0.4, and 0.2 respectively, and 0 (zero) is established in relation to other sectional images. It is determined in the twenty first aspect that the sectional image whose probability of including only the background is of the highest value, with the exception of the value of zero, and whose absolute value of the skewness of respective sectional images is less than the threshold, to be the sectional image whose probability of including only the background is high. By virtue of the matter described above, the sectional image can be selected whose higher probability of including only the background on the location of the image and giving it preference.

According to a twenty second aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the sectional image selection process is given beforehand a probability of including only the background respectively in each sectional image involved in a plurality of areas, selects the sectional image having the highest probability of including only the

CA 02249140 2001-10-23 background from respective areas among those sectional images whose absolute value of the skewness of the prescribed characteristic value is less than the threshold, thus determining the sectional image concerned as the sectional image whose probability of including only the background is high.

In the fourteenth aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including only the background is high, as in the process of the fourteenth aspect, and in addition thereto, the object detection and background removal method of the twenty second aspect, wherein probabilities of including only the background are given in every sectional image involved in a plurality of areas, selects a sectional image having the highest probability of including only a background from respective areas among those sectional images whose absolute value of the skewness of the brightness is less than thethreshold, thus determining the sectional image concerned to be the sectional image whose probability of including only the background is high.

Namely, the invention of the twenty second aspect establishes a plurality of a group of sectional images to which probabilities are given in comparison with the twenty first aspect. On account of this matter, it enables the sectional image to be selected while establishing the probabilities of including only the background to four corners of the image independently.

According to a twenty third aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the sectional image having a high probability of including only the background is selected from the sectional images involved in at least one area in the sectional image selection process.

CA 02249140 2001-06-05

In the fourteenth aspect to the twenties aspect of the invention, the object detection and background removal method selects the sectional image whose probability of including only the background is high, as in the process of the fourteenth aspect to the twenties aspect, and in addition thereto, the object detection and background removal method of the twenty third aspect selects a sectional image whose probability of including only a background is high from a sectional image involved in at least one area.

According to a twenty fourth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the statistic estimation process estimates a mean value and a standard deviation of the prescribed characteristic value overthe whole picture area while utilizing the mean value and the standard deviation of the prescribed characteristic value of the sectional image including only the background.

In the second aspect to the twenty third aspect of the invention, the object detection and background removal method estimates the statistics in the whole picture area from the sectional image including only the background, as in the process of the second aspect to the twenty third aspect, and in addition thereto, the object detection and background removal method of the twenty fourth aspect estimates a mean value and a standard deviation of the brightness and so forth of the background over the whole picture area while utilizing for example, the mean value and the standard deviation of the brightness of the sectional image including only the background.

Namely, the invention of the twenty fourth aspect, since the mean value and the standard deviation of the brightness and so forth of the sectional image including only the background selected previously are the mean value and the standard deviation of the brightness and so forth of the background of the location of the sectional image

CA 02249140 2001-06-05 concerned, thus estimates a mean value and a standard deviation of the brightness and so forth of a background over the whole picture due to the fact that the mean value and the standard deviation in the location of the sectional image are subjected to interpolation and extrapolation.

According to a twenty fifth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein when the statistics estimation process estimates a mean value and a standard deviation of the prescribed characteristic value for the background in the sectional image including an image with the exception of the background, if it exists that the mean value and the standard deviation of the prescribed characteristic value of the sectional image includes only the background located in the neighbourhood and the mean value and the standard deviation of the prescribed characteristic value of the sectional image includes images with the exception of the background estimated previously in the same neighbourhood, the mean value and the standard deviation is thus estimated by averaging above respective average values and the standard deviations, while if these do not exist, estimating the mean value and the standard deviation of the prescribed characteristic value in other sectional images including images with the exception of the background, estimation processing is repeated, both of a mean value and a standard deviation of the prescribed characteristic value of the background, until it is capable of estimating a mean value and a standard deviation of the prescribed characteristic value of the background in the whole of the sectional images, including images with the exception of the background.

In the twenty third aspect of the invention, the object detection and background removal method estimates the statistics in the whole picture area from the partial image including only the background, as in the process of the twenty third

CA 02249140 2001-06-05 aspect, and in addition thereto, in the object detection and background removal method of the twenty fifth aspect, when the statistics estimation process estimates, for instance, a mean value and a standard deviation of the brightness by way of the background in the sectional image including images with the exception of the background, if it exists that the mean value and the standard deviation of the brightness of the sectional image includes only the background located in the neighbourhood and the mean value and the standard deviation of the brightness of the sectional image including images with the exception of the background estimated previously in the same neighbourhood, the mean value and the standard deviation is estimated by averaging above respective average values and the standard deviations, while if these do not exist, estimating the mean value and the standard deviation of the brightness in other sectional images including images with the exception of the background, and repeating the estimation processing both of the mean value and the standard deviation of the brightness of the background until it is capable of estimating the mean value and the standard deviation of the brightness of the background in all of the sectional images, including images with the exception of the background.

Namely, the invention of the twenty fifth aspect estimates the mean value and the standard deviation of the brightness and so forth which the background would have, in the location of the sectional image including images with the exception of the background. For instance, in Figure 5, when the sectional image 2-A judged as the sectional image including image except the background, three of 1-A, 3-A, and 2-B within the sectional images of the neighbourhood having the side in common are judged as the sectional images including only the background. Consequently, it can be understood that the mean value and the standard deviation of the brightness of the background in the location of the sectional image 2-A analogize with those of the above sectional images 1-A, 3-A, and 2-B, therefore, a mean value and a standard deviation

CA 02249140 2001-10-23 of the brightness of the sectional image 2-A can be estimated by an equation (9) based on the mean vaiue and the standard deviation of the brightness of the sectional images

1-A, 3-A, and 2-B.

At2-A — ( jlil-A + JX3-A + fJ.2-3 )/3 σζ-Α = ( σι-Α '+ σ3-λ + σζ-_Β )/3 (9)

Similarly, in Figure 5, the sectional image 2-D which is judged as the partial image including image except the background, a mean value and a standard deviation thereof is estimated from the mean value and the standard deviation of the partial images 2-C, and 1-A which are judged as the sections] image including only the background. As described above, only the sectional image having a judged sectional image of including only the background in the neighbourhood thereof among sectional images judged as image including image except the background, to which the estimation of a mean value and a standard deviation of the brightness by way of the background is implemented. The sectional image obtaining the mean value and the standard deviation by the present estimation is dealt with by way of the sectional image including only the background thereafter, thus being utilized in estimation of another sectional image determined to be an image including images except the background. It is capable of estimating the mean value and the standard deviation of the brightness by way of the background over all of the whole sectional images, due to repeating of the present operation.

Namely, in the neighbourhood of the sectional image i-p, when there is only one sectional image whose mean value and standard deviation of the brightness and so forth for the background is estimated and whose number of the sectional image is

CA 02249140 2001-06-05 set to be j-q, a mean value and a standard deviation of the brightness and so forth by way of a background in the sectional image i-p are estimated by an equation (10).

JIi-p= JX j-q

O*i-p“ O’j-q (10)

Similarly, when the mean value and the standard deviation of the brightness are estimated in the neighbourhood of the two sectional images j-q and k-r, a mean value and a standard deviation of the brightness and so forth by way of a background in the sectional image i-p are estimated by an equation (11).

jXi-p — (jXj-q + /Xk-r) /2

O^ri“P “ ( O’j-q + O^pk-r) /2 (11)

Further, when the mean value and the standard deviation of the brightness are estimated in the neighbourhood of the three sectional images j-q, k-r, and l-s, a mean value and a standard deviation of the brightness and so forth by way of a background in the sectional image i-p are estimated by an equation (12).

#ii-p = + +_μιρ /3 σι-ρ = (aj-q + ak-r +σι-ε) /3 (12)

And so forth, it is capable of being defined along the number of the sectional images in the neighbourhood thereof.

In the above estimation processing, there is defined the sectional image having a side in common in regard to the sectional image by way of the target, to be a neighbourhood, in addition thereto, it is capable of defining neighbourhood of including sectional image having the vertexes in common, and including a sectional image whose distance is long, as the neighbourhood, thus it is capable of implementing the same

CA 02249140 2001-06-05 estimation processing as above. Further, the equations (10), (11), (12) for estimating a mean value and a standard deviation are simple averages, however, it is capable of utilizing complicated equations such as an equation taking a center value, an equation weighted-averaging by distance of sectional images therebetween, and an equation using a quadratic average, or a cubic average.

According to a twenty sixth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, the statistic estimation process calculates the location of the center of gravity of the sectional image including images with the exception of the background, to be taken as the location of the center of gravity of an object, and calculating the distances between respective sectional images and the location of the center of gravity of the object. When the statistic estimation process estimates a mean value and a standard deviation of the prescribed characteristic value of the background in the sectional images, including images with the exception of the background, in cases where there exists only one set of a mean value and a standard deviation of the sectional image which is located farther than the distance between the sectional image, including images with the exception of the background and the location of the center of gravity, and which is located in the neighbourhood thereof including only the background, the mean value and the standard deviation concerned are estimated by obtaining a mean value thereof, while in cases where there exists more than one set of a mean value and a standard deviation of the sectional image, estimating the mean value and the standard deviation of the prescribed characteristic value in other sectional images, including images with exception of the background, estimating processing of a mean value and a standard deviation of the prescribed characteristic value of the background is repeated until it is capable of estimating a mean value and a standard deviation of the prescribed

CA 02249140 2001-06-05 characteristic value of a background in all of the whole sectional images, including images with the exception of the background.

In the twenty fourth aspect of the invention, the object detection and background removal method estimates the statistics in the whole picture area from the sectional image including only the background, as in the process of the twenty fourth aspect, and in addition thereto, in the object detection and background removal method of the twenty sixth aspect, the statistic estimation process calculates the location of the center of gravity of the sectional image including the image with the exception of the background, to be taken as the location of the center of gravity of an object, and calculates the distance between respective sectional images and the location of the center of gravity of the object. When the statistic estimation process estimates a mean value and a standard deviation of the brightness by way of the background in the sectional images, including images with the exception of the background, in cases where there exists only one set of a mean value and a standard deviation of the sectional image which is located further than the distance between the sectional image, including images with the exception of the background and the location of the center of gravity, which is located in the neighbourhood thereof including only the background, the mean value and the standard deviation concerned are estimated by obtaining a mean value thereof, while in cases where there exists more than one set of a mean value and a standard deviation of the sectional image, estimating the mean value and the standard deviation of the brightness in other sectional images, including images with exception of the background, estimating processing of a mean value and a standard deviation of the brightness of the background is repeated until it is capable of estimating a mean value and a standard deviation of the brightness of a background in all of the whole sectional images except the background.

CA 02249140 2001-10-23

Namely, the invention of the twenty sixth aspect obtains the location of the center of gravity of an object to be a detection target while calculating the location of the center of gravity of the sectional image except the background in comparison with the twenty fifth aspect, thus estimating a mean value and a standard deviation of the brightness by way of the background of the sectional image except the background by using only the mean value and the standard deviation of the brightness of the background in the sectional image further away from the location of the center of gravity among the sectional images including only the background in the neighbourhood thereof.

For example, imagine that a brightness distribution of a sectional image analogizes with that of another sectional image on the inside of background and an object of detection target accidentally. In this case, the processing of an interpolation / extrapolation to the sectional image of the neighbourhood as described in the twenty fifth aspect cannot estimate correctly the mean value of the brightness of the background at the intermediate location between the sectional image at the location of background and the sectional image within the object. The invention of the twenty sixth aspect can estimate a mean value of the brightness of the background correctly in this case using Figure 6.

As shown in Figure 6, the brightness distribution of the background area 2 analogize with that of inside of the object 10, thus the sectional image determined to include only the background appears at both of the background area 2 and the inside of the object 10. However, the sectional image which passes an object-contours 9 is determined as a sectional image except the background because the standard deviation or the skewness becomes larger. In Figure 6, the sectional image with light hatching is the sectional image determined to include only the background. When there is obtained the mean value of the brightness by way of the background at the location of the sectional

CA 02249140 2001-10-23 image 11 existing on the object-contours 9, in accordance with the twenty fifth aspect, from only the sectional images 12, 13, 18, and 19 which are judged that only the background and exists in the neighbourhood thereof, there is obtained a wrong intermediate value of the background area 2 and the inside of the object 10.

The twenty sixth aspect can handle such a case because there is calculated the center of gravity of the sectional image judged as except the background to be obtained the location of the center of gravity. The twenty sixth aspect compares the distance between the sectional images 12 to 19 in the neighbourhood thereof and an object center location 20 to be a rough location of the object with the distance between the watched sectional image 11 and the object center of gravity location 20, subsequently, setting only the sectional images 12, and 13 located farther away therefrom to be candidates of calculation, thus eliminating the influence of the sectional image wrongly determined to be the background regardless of its existence within the object 10. Since the sectional image which passes the object-contours 9 is regarded as except the background, the location of the center of gravity viewing from the watched sectional image 11 cannot be obtained accurately. However there is no influence.

The influence of the sectional image wrongly determined as only background in the inside of the object remains in the background estimated previously. Where it is intended to detect an object based on the background estimated previously thereafter, the influence can be eliminated easily by taking account of connectivity to the sectional image having a probability of including only the background.

According to a twenty seventh aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the statistic estimation process is provided with respective peculiar neighbourhood relationships in terms of respective sectional images on an image.

CA 02249140 2001-06-05

In the twenty fifth aspect of the invention, the object detection and background removal method estimates the statistics in the whole picture area from those sectional images including only the background, as in the process of the twenty fifth aspect, and in addition thereto, the twenty seventh aspect gives respective peculiar neighbourhood relationships in terms of the respective sectional images on the image.

Namely, the invention of the twenty seventh aspect does not use the object center of gravity location as described in the twenty sixth aspect, but rather the neighbourhood relationship of the sectional image is set in every respective sectional image of the image beforehand. Figure 7 is an explanation view showing the neighbourhood relationship. When there is estimated a mean value and a standard deviation of the brightness of the background in the sectional image 11, thus determining beforehand which of the sectional images 12 to 19 have the side and the apexes in common is defined as the neighbourhood.

As shown in Figure 8, when the detection target object 3 is photographed at the left corner of the image, it does not necessarily need the location of the center of gravity. When there is estimated the mean value and the standard deviation of the brightness of the background in the sectional image 11 in Figure 8, defines the sectional images 13,14, and 16 in Figure 7 to be the neighbourhood. On account of this matter, it is capable of using only the statistics of the sectional image located in the direction of apparently the background, and it is capable of obtaining object contours accurately.

Similarly, as shown in Figure 9, there is supposed that the detection target object 3 exists beforehand below the center of the picture. When there is taken notice of the sectional image 11 in Figure 9, there does not occur inversion of relationship of the location between the inside of object and the background because only the sectional images 12, 13, and 15 in Figure 7 at the left side area of the boundary line 21 are treated as the neighbourhood, and because only the sectional images 13, 14, and 16

CA 02249140 2001-10-23 in Figure 7 at the right side area of the boundary line 21 are treated as the neighbourhood, there does not occur a wrong estimation in the neighbourhood of the boundary. In the person photographs, similar composition is in use in lots of cases, the twenty seventh aspect is effective.

According to a twenty eighth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the statistic estimation process judges the mean value and the standard deviation of the prescribed characteristic value in the sectional images which include only the background as the mean value, and the standard deviation of the prescribed characteristic value of the background in the location of a center pixel of the sectional image, when estimating a mean value and a standard deviation of the prescribed characteristic value of the background in respective pixels on the image, and calculating the distances between the location of the pixel and the center pixel of the sectional images which include only the background, thus estimating a mean value and a standard deviation of the prescribed characteristic value of the background in the location of the pixel. The mean value and standard deviation of the prescribed characteristic value in the location of center pixel of the sectional image which include only the background are weighted in relation to the corresponding distance between the location of the pixel and the center pixel of the sectional images which include only the background.

In the twenty fifth aspect to the twenty seventh aspect of the invention, the object detection and background removal method estimates the statistics in the whole picture from the sectional image, however, the object detection and background removal method of the twenty eighth aspect implements an estimation in every pixel.

CA 02249140 2001-06-05

The invention of the twenty eighth aspect is explained using Figures 5 and 10. In

Figure 5, the sectional image with light hatching includes only the background, thus a set of these partial images is taken to be Ψ. Further, a center pixel location of the sectional image group is specified by c1-A, d-Β, ..., c9-l and column and row. The mean value and the standard deviation of the brightness of the background in the center pixel location agrees with the mean value and the standard deviation of the brightness of the sectional image concerned.

When the mean value and the standard deviation of the brightness of the background in the estimated pixel location 22 is estimated, the distance between the present estimated pixel location 22 and the center pixel location in the set Ψ of the sectional image which include only the background is calculated. Here, the coordinates location of the estimated pixel location 22 is set to be (Xq , yq) _f and the center coordinates location ci-p of the sectional image i-p which includes only the background is set to be (xci-_P/ y_ci-_P) , thus the distance d_q,ci-_P ofthe two points therebetween is calculated as equation (13).

dq,ci-p = V { (Xq—Xci-p)² + (yq—yci-p)²} (13)

A mean value and a standard deviation of the brightness of the background in the estimated picture element location 22 is estimated based on the equation (14), using the distance d_q,ci-_P.

{ Σ i-peV jXi-p (dq,ci-p) ²) / { Σ i-_Pe Ψ ( dq, ci-p ) “² } <Tq⁼ { Σ i-pe Ψ (Ti-p (dq,ci-p) ²) / { Σ i-p G Ψ ( dq, c i-p ) ~² } (14)

Here, the denominator is the term for normalization. Consequently, in the equation (14), the estimation is implemented in such a way that it is weighted to be averaged in proportion to the squared reciprocal of distance. There is obtained the mean value and the standard deviation of the brightness of the background in terms

CA 02249140 2001-06-05 of the whole pixels by repeating the above procedure. It is capable of being used as a reciprocal or a cube of reciprocal for a method of adding weight.

According to a twenty ninth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the statistic estimation process judges a location of the center of gravity of the sectional image which includes only the background as the location of the center of gravity of an object, when estimating a mean value and a standard deviation of the prescribed characteristic value of the background in respective pixels on an image, thus assuming a straight line connecting the pixel and the location of the center of gravity of the object, and a half straight line located at opposite side of the location of the center of gravity of the object from the pixel on the straight line, and subsequently, selecting whole center pixels of the sectional image which include only the background, which sectional image intersected location is located on the half straight line while dropping a perpendicular to the straight line from the center pixel of the sectional image which includes only the background, then, calculating a distance between the pixel and the location of the center pixel of the sectional image which includes only the background, thus implementing estimation of the mean value and the standard deviation of the prescribed characteristic value of the background at the location of the pixel on the image. The mean value and standard deviation of the prescribed characteristic value of the background at the location of the center pixel in the sectional image which includes only the background are then weighted to average in relation to the corresponding distance.

In the twenty fourth aspect of the invention, the object detection and background removal method estimates the statistics in the whole picture from the sectional image which includes only the background, as in the process of the twenty

CA 02249140 2001-06-05 fourth aspect, and in addition thereto, in the object detection and background removal method ofthe twentieth ninth aspect, the statistic estimation process judges the location of the center of gravity of the sectional image which includes only the background as the location of the center of gravity of an object, when estimating a mean value and a standard deviation ofthe brightness ofthe background in respective pixels on an image, thus assuming a straight line connects the pixel and the location of the center of gravity of the object, and a half straight line located at opposite sides of the location of the center of gravity of the object from the pixel on the straight line, and subsequently, selecting whole center pixels of the sectional image which includes only the background, which sectional image intersected location is located on the half straight line while dropping a perpendicular to the straight line from the center pixel of the sectional image which include only the background, then, calculating the distance between the pixel and the location of the center pixel of the sectional image which includes only the background, thus implementing estimation of a mean value and a standard deviation ofthe brightness ofthe background at the location ofthe pixel on the image. The mean value and standard deviation ofthe brightness ofthe background ofthe location ofthe center pixel in the sectional image which includes only the background are then weighted to average in relation to the corresponding distance.

Namely, the invention of the twenty ninth aspect utilizes only the mean value and the standard deviation ofthe brightness ofthe sectional image which includes only the background, which the mean value and the standard deviation are located at the opposite side of the object center of gravity location when there is obtained a mean value and a standard deviation of the brightness ofthe background of a pixel location, in comparison with the twenty eighth aspect. The invention of the twenty ninth aspect will be described referring to Figure 10.

CA 02249140 2001-06-05

Firstly, the object center of gravity location 20 is obtained while calculating the center of gravity location of the sectional image which includes only the background. When the mean value and the standard deviation of the brightness of the background is obtained in the estimated pixel location 22, it is assumed that a straight line 23 connects the estimated pixel location 22 and the object center of gravity location 20.

There is dropped the straight line 24 to be the perpendicular line from the center pixel location d-Β of the sectional image to the straight line 23 while taking notice of 1-B of the sectional images which include only the background. Since the intersection point of the straight line 23 and the straight line 24 belongs to a half straight line located at an opposite side of the object center of gravity location 20 from the estimated pixel location 22 of the straight line 23, putting it into the set Ψ of the sectional image for utilizing in case of estimation of the sectional image 1-B. Above operation is repeated in terms of the whole sectional images which include only the background, before implementing the estimation processing based on the equation (14).

The invention of the twenty ninth aspect repeats the present estimation processing over all of the pixels.

According to a thirtieth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein the statistic estimation process is provided with a location of the center of gravity of an object which comes to be the center of gravity of an object beforehand.

In the twenty sixth aspect to the twenty ninth aspect of the invention, the object detection and background removal method estimates the statistics in the whole picture area from the sectional image which includes only the background, as in the process of the twenty sixth aspect to the twenty ninth aspect, and in addition thereto, in the object detection and background removal method of the thirtieth aspect, the

CA 02249140 2001-10-23 object center of gravity location to be the center of gravity of the object is given beforehand.

According to a thirty first aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein in regard to isolating the object of detection target at the comparison process, when the comparison process also implements threshold processing in the respective location of the pixels, and a first threshold is calculated to be defined in such a way that a constant set beforehand is multiplied by the standard deviation of the prescribed characteristic value of the background estimated previously, then the above multiplied number is subtracted from a mean value of the prescribed characteristic value of the background estimated previously. Secondly a second threshold is calculated to be defined in such a way that a constant also set beforehand is multiplied by the standard deviation of the prescribed characteristic value of the background estimated previously, then the above multiplied number is added to a mean value of the prescribed characteristic value of the background estimated previously, thus in cases where the prescribed characteristic value of the location of the pixel is larger than the first threshold and is smaller than the second threshold, and determining the pixel as the background, the comparison process removes the background and isolates the object due to the fact that the comparison process causes the same processing to be executed over all of the pixels.

In the second aspect to the thirtieth aspect of the invention, the object detection and background removal method determines the threshold over the whole picture area from the statistics estimated previously, and subsequently, compares the determined threshold over the whole picture area with the input image, thus isolating only the detection target object, as in the process of the second aspect to the thirtieth

CA 02249140 2001-10-23 aspect, and in addition thereto, in the object detection and background removal method of the thirty first aspect, in regard to isolation of the object of detection target at the comparison process, when the comparison process implements threshold processing in the respective location of the pixels, a first threshold is calculated to be defined in such a way that a constant set beforehand is multiplied by the standard deviation of the brightness of the background estimated previously, then the above multiplied number is subtracted from a mean value of the brightness of the background estimated previously. Secondly a second threshold is calculated to be defined in such a way that also a constant set beforehand is multiplied by the standard deviation of the brightness of the background estimated previously, then the above multiplied number is added to a mean value of the brightness of the background estimated previously, thus in cases where the brightness of the location of the pixel is larger than the first threshold and is smaller than the second threshold, thus determining the pixel as the background, the comparison process removes the background to isolate the objet due to the fact that the comparison process causes the same processing to be executed over all of the pixels.

Namely, the invention of the thirty first aspect removes only the background area, to isolate the object to be the detection target using contours by calculating the threshold based on the mean value and the standard deviation of the brightness of the background estimated previously from the input image.

The brightness values of the background scatter in the vicinity of the mean value of the brightness. The size of dispersion is capable of being estimated by the standard deviation. According to table 1 of normal distribution function described in the literature : Mathematical Statistics written by T. Takeuchi, published by Toyo Keizai, 1963, pp. 361, there can be read the probability that the slipping off from the mean value disperses more than three times of the standard deviation is 0.26%. There is expressed in different words, when the brightness value of the background is in a state

CA 02249140 2001-06-05 of the normal distribution, 99.74% of the pixel in the whole picture elements belong to the inside of three times of a mean value ± a standard deviation.

Consequently, the first threshold forjudging whether or not a pixel located at coordinates position (x, y) is a background is calculated by an equation (15) using a constant a - , and whose value is determined beforehand as positive or zero, a mean value in the sectional image i-p including the pixel location, and the standard deviation σι - p .

ri,x,_y = μι-p—α-σι-ρ (15)

Similarly, the second threshold is calculated by an equation (16) using a constant a + , and whose value is positive or zero, and also determined beforehand.

T2,x,y = μΐ-ρ-α+CTi’P (¹⁶)

It becomes possible to distinguish the pixels constituting the background due to the threshold processing of equation (17) by utilizing the above two thresholds.

Ti,x,y^Ix,y^UM (17)

The invention of the thirty first aspect is described by using the arithmetic mean or the standard deviation, however it is also capable of constituting the invention of the aspect by using a geometric mean, a harmonic mean, a median and so forth which statistically estimate amounts to be a representative value, and the statistics representing dispersion such as an absolute deviation, and a quarter deviation described in the literature : Encyclopaedia of Mathematical Sciences published by Maruzen Co., Ltd, 1991, pp. 495.

According to a thirty second aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as contours is concerned, wherein in regard to isolation of the object of detection target at the comparison process, the comparison process implements a threshold processing over all of the pixels, to detect

CA 02249140 2001-10-23 areas linked to one another by way of areas to be a candidate of a background, and determines these areas concerned as areas which include the most number of sectional images having a probability of including only the background.

In the thirty first aspect of the invention, the object detection and 5 background removal method determines the threshold in the whole picture area from the statistics described above, and subsequently, compares the threshold in the whole picture area with the input image, and isolates the object to be the detection target, as in the process of the twenty first aspect, and the object detection and background removal method of the thirty second aspect implements the threshold processing over all of the pixels, before detecting the areas linked with each other by way of the area to be a candidate of the background, and removes the background to isolate the object due to the fact that there is only removed the candidate area which includes the largest number of sectional images having a probability of including only the background, to be the background area from among all the candidate areas of the background.

Namely, the invention of the thirty second aspect removes the background to isolate the object due to the fact that there is only removed the candidate area which includes the largest number of sectional images having a probability of including only the background. In virtue of this matter, even though when the brightness distribution of the background is close to the brightness distribution of the inside of object, it prevents a part of the inside of the object from being wrongly removed as the background.

According to a thirty third aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately as far as an outline is concerned, wherein in regard to isolating the object of detection target at the comparison process, the comparison process implements a threshold processing over all of the pixels, to

CA 02249140 2001-10-23 detect areas linked to one another by way of areas, to be a candidate of a background, and determines these areas concerned as areas which include the most number of sectional images having a probability of including only the background from among ali candidate areas of the background.

In the thirty first aspect of the invention, the object detection and background removal method determines the threshold in the whole picture area from the statistics described above, and subsequently, compares the threshold in the whole picture area with the input image, and isolates the object to be the detection target, as in the process of the twenty first aspect concerned. The object detection and background removal method of the thirty second aspect also implements threshold processing over all of the pixels before detecting the areas linked with each other by way of the area to be the candidate of the background, and removes the background to isolate the object, due to the fact that there is only removed the candidate area which includes the largest number of sectional images which include only the background, to be the background area from among all the candidate areas of the background. According to a thirty fourth aspect of the invention, there is provided a method for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein in regard to isolating the object of detection target at the comparison process, the comparison process implements a threshold processing over all of the pixels, to detect areas linked to one another by way of areas to be a candidate of a background, and determines the area which includes the smallest number of sectional images which include image with the exception of the background, from among all the candidate areas of the background, as a background area, so that the comparison process removes the background to isolate the object.

In the thirty first aspect of the invention, the object detection and background removal method determines the threshold in the whole picture area from

CA 02249140 2001-10-23 the statistics described above, and subsequently, compares the threshold in the whole picture area with the input image, and isolates the object to be the detection target, as in the process of the twenty first aspect concerned, and the object detection and background removal method of the thirty second aspect implements the threshold processing over all of the pixels, before detecting the areas linked with each other by way of the area to be the candidate of the background, and removes the background to isolate the object. There is only removed the candidate area which includes the smallest number of sectional images except the background, from among all the candidate areas of the background, to be a background area.

According to a thirty fifth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely’and accurately using contours, comprising a sectional image statistic calculation means for calculating a mean value and a standard deviation of the prescribed characteristic value of the sectional image while dividing to be processed an input image into sectional images, a background sectional image selection means which determines the sectional image whose standard deviation is of the smallest value in the sectional images as a sectional image having a probability of including only a background, and subsequently, compares the standard deviation of the prescribed characteristic value of the sectional image with a standard deviation of the prescribed characteristic value of other sectional images, and determines the sectional image having a standard deviation wherein the difference between the standard deviation concerned and another standard deviation is less than a threshold to be the sectional image which includes only the background, a background statistic estimation means for investigating all of the mean values and standard deviations in the sectional images which include only the background and in other sectional images, further in the sectional images including images with the exception of said

CA 02249140 2001-10-23 background and in the sectional images including only the background located in the neighbourhood of the sectional image, and in the sectional image by way of the background estimated previously in another sectional image, and a threshold generation object detection and background removal means wherein in order to isolate an object to be removed background by using the mean value and the standard deviation in the whole sectional images, a second threshold is calculated to be defined in such a way that a constant set beforehand is multiplied by the standard deviation of the prescribed characteristic value of the background estimated previously, then the above multiplied number is added to a mean value of the prescribed characteristic value of the background estimated previously, and subsequently, calculates it all over the pictures to be outputted, thus determining pixels within the threshold as a background while using said two kinds of thresholds and determining pixels without the threshold as an object of detection target.

According to a thirty sixth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, with an image constituted by a virtually constant background and an object of detection target, the device roughly consisting of a sectional image statistic calculation means, a background sectional image selection means, a background statistic estimation means, and a threshold generation object detection and background removal means, the sectional image statistic calculation means comprising a sectional image division means for dividing input images into sectional images, a mean value and a standard deviation calculation means which calculates a mean value and a standard deviation of the prescribed characteristic value to be outputted in each respective sectional image with the sectional image signals as inputs, and a sectional image statistic storage means storing, to be outputted later, the mean value and the standard deviation of the

CA 02249140 2001-10-23 prescribed characteristic value of each respective sectional image with the mean value and the standard deviation of the prescribed characteristic value of the sectional images as inputs, said background sectional image selection means comprising a minimum standard deviation reference background only sectional image selection means for outputting a sectional image whose standard deviation of the prescribed characteristic value is of the smallest value among sectional images, as a sectional image whose probability of including only a background with the mean value and the standard deviation of the prescribed characteristic value of the sectional image, a background only sectional image selection means comparing a standard deviation of the prescribed characteristic value of a sectional image whose probability of including only the background with the standard deviation of the prescribed characteristic value in other sectional images, and outputs a partial image where the standard deviation wherein the difference between the standard deviation concerned and a standard deviation of the prescribed characteristic value of a sectional image whose probability of including only the background, is less than a threshold, as a sectional image which includes only the background, and a background only sectional image statistic storage means for storing a location of the sectional image which includes only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned to output them at any time, the background statistic estimation means comprising a background exception sectional image selection means, when a command for investigating a sectional image including image with the exception of a background comes thereto, and investigating both of the mean value and the standard deviation of the prescribed characteristic value in a sectional image which inciudes only a background and the mean value and the standard deviation of the prescribed characteristic value by way of an estimated background in other sectional images, if there exists a sectional image having no estimated value of the mean value

CA 02249140 2001-06-05 and the standard deviation of the prescribed characteristic value of a background exists, outputting the partial image concerned. In cases where the mean value and the standard deviation of the prescribed characteristic value are estimated with regard to all of the sectional images, the background statistic estimation means issues a command to generate a threshold for the sake of object detection and background removal, a neighbourhood background only sectional image existence judgement means investigates mean values and standard deviations both of sectional images which include images with the exception of a background, and sectional images including only a background located in the neighbourhood of the sectional images, and also investigates mean values and standard deviations of the prescribed characteristic value of a background in other sectional images. Where there exists a sectional image having only one set of a mean value and a standard deviation of the prescribed characteristic value estimated in the neighbourhood thereof, a command is issued so as to estimate a mean value and a standard deviation of the prescribed characteristic value of a sectional image which includes images with the exception of the background. A mean value and a standard deviation interpolation/extrapolation means, when receiving a command to estimate a mean value and a standard deviation of the prescribed characteristic value in the sectional image including image with the exception of the background, estimates both the mean values and standard deviations of the prescribed characteristic value of a sectional image which includes only the background in the neighbourhood thereof by averaging, then outputting an estimated sectional image selection command signal so as to select the next sectional image, and an estimated statistic storage means stores the mean value and the standard deviation of the prescribed characteristic value estimated previously to be outputted. The threshold generation object detection and background removal means comprises a threshold generation means, wherein when a command for calculating a threshold is entered after

CA 02249140 2001-10-23 completing whole mean values and standard deviations of the prescribed characteristic value in all of the sectional images, by using the mean value and the standard deviation in all of the sectional images, a second threshold is calculated to be defined in such a way that a constant set beforehand is multiplied by the standard deviation of the prescribed characteristic value of the background estimated previously, then the above multiplied number is added to a mean value of the prescribed characteristic value of the background estimated previously, and subsequently, calculates it over all the pictures to be outputted, and a threshold processing means determines pixels within the threshold as a background, while using the two kinds of thresholds, and determines pixels without the threshold as an object of detection target.

According to a thirty seventh aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the prescribed characteristic value is at least one of a brightness, a colour information, and an edge information.

According to a thirty eighth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a minimum standard deviation reference background only sectional image selection means to output partial images in such a way that it causes the specified number of sectional images having a smaller number than the standard deviation of the prescribed characteristic value to be outputted in order, while taking such sectional images to be the sectional image whose probability of including only the background with the mean value and the standard deviation of the prescribed characteristic value of the sectional image as inputs, a background only sectional image selection means compares the standard deviation of the prescribed

CA 02249140 2001-10-23 characteristic value of the sectional image having a probability of including only the background with the standard deviation of the prescribed characteristic value of other sectional images, thus outputting the sectional image having a standard deviation wherein the difference between the standard deviation concerned and the standard deviation of the prescribed value of the sectional image having a high probability of including only the background, as the sectional image which includes only a background, and a background only sectional image statistic storage means stores the location of the sectional image which includes only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, and outputs them.

According to a thirty ninth aspect of the invention, there is provided a device for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means further comprises a standard deviation difference background only sectional image selection means which selects to be outputted the sectional image having a standard deviation is of the nearest most value to the standard deviation of the prescribed characteristic value instructed beforehand. A background only sectional image selection means compares the standard deviation of the prescribed characteristic value of the sectional image having a probability of including only the background, with the standard deviation of the prescribed characteristic value of other sectional images, and selects, to be outputted later, the sectional image having a standard deviation wherein the difference between the standard deviation concerned and the standard deviation of the prescribed value of the sectional image having a probability of including only the background is high as the sectional image which includes only a background, and a background only sectional image statistic storage means stores the location of the sectional image which includes only the background

CA 02249140 2001-06-05 and the mean value and the standard deviation of the prescribed characteristic vafue of the sectional image concerned, and outputs them.

According to a fortieth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a standard deviation reference background only sectional image selection means, which outputs a specified number of sectional images, in order, having the nearest value to the standard deviation of the skewness of the prescribed characteristic value instructed beforehand, and judges the sectional image concerned as the sectional image having a high probability of including only the background. A background only sectional image selection means compares the standard deviation of the prescribed characteristic value of the sectional image having a probability of including only the background with the standard deviation of the prescribed characteristic value of other sectional images, and judges, to be outputted later, the sectional image having a standard deviation wherein the difference between the standard deviation concerned and the standard deviation of the prescribed value of the sectional image having a probability of including only the background is high, as the sectional image which includes only a background, and a background only sectional image statistic storage means stores the location of the sectional image which includes only the background, and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, and outputs them.

According to a forty first aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a mean value and standard deviation reference background only sectional image selection means for outputting the sectional

CA 02249140 2001-10-23 image having a standard deviation which is of the most nearest value to the mean value and the standard deviation of the prescribed characteristic value instructed beforehand, as the sectional image having a probability of including only a background. A background only sectional image selection means compares the standard deviation of the prescribed characteristic value of the sectional image having a probability of including only the background, with the standard deviation of the prescribed characteristic value of other sectional images, and outputs the sectional image having a standard deviation wherein the difference between the standard deviation concerned and the standard deviation of the prescribed value of the sectional image whose probability of including only the background is high as the sectional image which includes only a background, and a background only sectional image statistic storage means stores the location of the sectional image which includes only the background, and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, and outputs them whenever necessary.

According to a forty second aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using an outline, wherein said background sectional image selection means comprises a mean value and standard deviation reference background only sectional image selection means which outputs sectional images as many as the number specified in order of nearer number to the mean value and the standard deviation of the prescribed characteristic value instructed beforehand as the sectional image having a probability of including only a background. A background only sectional image selection means compares the standard deviation of the prescribed characteristic value of the sectional image having a probability of including only the background, with the standard deviation of the prescribed

CA 02249140 2001-10-23 characteristic value of other sectional images, and selects, to be outputted later, the sectional image having a standard deviation wherein the difference between the standard deviation concerned and the standard deviation of the prescribed value of the sectional image whose probability of including only the background is high, as the sectional image which includes only a background, and a background only sectional image statistic storage means stores the location of the sectional image which includes only the background, and the mean value, and the standard deviation of the prescribed characteristic value of the sectional image concerned, and outputs them whenever necessary.

According to a forty third aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means selects the sectional image whose probability of including only a background is high from sectional image involved in an area.

According to a forty fourth aspect of the invention, there is provided a device of object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means selects the sectional image whose probability of including only a background is high from sectional images involved in a plurality of areas.

According to a forty fifth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, with an image having a virtually constant background and an object of detection target, the device roughly consisting of four means: a sectional image statistic calculation means, a background sectional image selection means, a background statistic estimation means, and a

CA 02249140 2001-10-23 threshold generation object detection and background removal means. The sectional image statistic calculation means comprises a sectional image division means performing division output of an input image into sectional images, a mean value and standard deviation and skewness calculation means calculates to be outputted a mean value, a standard deviation, and a skewness of a prescribed characteristic value in every respective sectional image, with the sectional image signal as inputs. A sectional image statistic storage means stores, to be outputted later, the mean value, the standard deviation, and the skewness of the prescribed characteristic value of each respective sectional image with the mean value, the standard deviation, and the skewness of the prescribed characteristic value of the sectional images as inputs, and the background sectional image selection means comprises a skewness threshold and minimum standard deviation reference background only sectional image selection means outputs the sectional image whose absolute value of the skewness is less than a threshold given beforehand in sectional images, and whose standard deviation of the prescribed characteristic value is of the smallest value, as the sectional image having a probability of including only a background. A background only sectional image selection means outputs the sectional image having a standard deviation wherein the difference between the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is high and the standard deviation of the prescribed characteristic value in the sectional images is less than the threshold, and outputs the sectional image having an absolute value of the skewness less than the threshold as the sectional image which includes only a background. A background only sectional image statistic storage means stores the location of the sectional image which includes only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, and outputs them and the background statistic estimation means comprises

CA 02249140 2001-06-05 a background exception sectional image selection means, when a command for investigating sectional images including images with the exception of a background, investigating a mean value and a standard deviation of the prescribed characteristic value of the sectional image including only the background and a mean value and a standard deviation of the prescribed characteristic value of estimated background of other sectional images, and if there exists a sectional image whose mean value and standard deviation of the prescribed characteristic value of the background is not estimated, outputting the sectional image concerned, and if the mean value and the standard deviation of the prescribed characteristic value of the background in respect to all of the sectional images are instructed, issues a command so as to generate a threshold for the sake of object detection and background removal, and a neighbourhood background only sectional image existence judgement means investigates all of mean values and standard deviations of the prescribed characteristic values, both of sectional images including images with the exception of backgrounds and sectional images including only background located in the neighbourhood of the sectional images, and mean values and standard deviations of the prescribed characteristic value of the estimated background of other sectional images, and when there exists only one sectional image whose mean value and standard deviation of the prescribed characteristic value in the neighbourhood, issues a command to estimate the mean value and the standard deviation of the prescribed characteristic value of the sectional image including images with the exception of the background, and when there exists no sectional image whose mean value and standard deviation of the prescribed characteristic value are estimated, issues a command to select the next sectional image, a mean value and standard deviation interpolation/extrapolation means, when receiving a command for estimating a mean value and a standard deviation of the prescribed characteristic value in the sectional image including images with the

CA 02249140 2001-06-05 exception of the background, thus estimates to be outputted, by averaging, the mean value and the standard deviation of the prescribed characteristic value of the sectional image including only the background in the neighbourhood, and by averaging the mean value and the standard deviation ofthe prescribed characteristic value of the estimated background of the sectional image in the neighbourhood thereof, simultaneously outputs an estimated sectional image selection command signal to select the next sectional image, and an estimated statistic storage means stores the mean value and the standard deviation ofthe prescribed characteristic value estimated previously; the threshold generation object detection and background removal means comprises a threshold generation means, which when receiving a command for calculating the threshold after the mean value and the standard deviation of the prescribed characteristic value of the background in all ofthe sectional images has been estimated, a first threshold is obtained in such a way that it causes the standard deviation multiplied by a constant to be subtracted from the mean value, by using the mean value and the standard deviation ofthe prescribed characteristic ofthe estimated background in all of the sectional images, for the sake of detecting an object and removing background, and a second threshold is obtained in such a way that it causes the standard deviation multiplied by a constant to be added to the mean value. Thus the first and the second threshold are calculated over the whole picture area to be outputted, and a threshold processing means takes pixels involved between the two thresholds to be the background, and taking other pixels to be an object of detection target by using the two thresholds.

According to a forty sixth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold background only

CA 02249140 2001-06-05 sectional image selection means which judges the sectional image whose absolute value of skewness is less than the threshold, from among the sectional images, as the sectional image whose probability of including only a background with the mean value and the standard deviation of the prescribed characteristic value of the sectional image and the skewness of the sectional image as inputs. A background only sectional image selection means outputs a sectional image having a standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is less than the threshold and whose absolute value of skewness is less than the threshold, as the sectional image including only the background, while comparing the standard deviation of the prescribed characteristic value of the sectional image whose probability of including the background, with the standard deviation of the prescribed characteristic value in other sectional images, and a background only sectional image statistic storage means stores the mean value and the standard deviation of the prescribed characteristic value of a location of the sectional image including only the background and the sectional image concerned and outputs them whenever necessary.

According to a forty seventh aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold and minimum standard deviation reference background only sectional image selection means which outputs sectional images whose absolute value of skewness is less than the threshold and wherein the number specified is in the order of having a smaller value than the standard deviation of the prescribed characteristic value, as a sectional image having a probability of including only a background, with the mean value and the standard

CA 02249140 2001-06-05 deviation of the prescribed characteristic value of sectional images and the skewness of sectional images as inputs, a background only sectional image selection means outputs the sectional image having a standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background, is less than the threshold and whose absolute value of skewness is less than the threshold, as the sectional image including only the background while comparing the standard deviation of the prescribed characteristic value of the sectional image whose probability of including the background with the standard deviation of the prescribed characteristic value in other sectional images; and a background only sectional image statistic storage means stores the mean value and the standard deviation of the prescribed characteristic value of a location of the sectional image including only the background and the sectional image concerned and outputs them.

According to a forty eighth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold and standard deviation reference background only sectional image selection means outputs a sectional image whose absolute value of skewness is less than the threshold value, and whose standard deviation is of the nearest value to the standard deviation of the prescribed characteristic value from among sectional images, as the sectional image having a probability of including only the background, with the mean value and the standard deviation of the prescribed characteristic value of sectional images and the skewness of sectional images as inputs; a background only sectional image selection means outputs the sectional image having a standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed

CA 02249140 2001-06-05 characteristic value of the sectional image whose probability of including only the background is less than the threshold, and whose absolute value of skewness is less than the threshold, as the sectional image including only the background while comparing the standard deviation of the prescribed characteristic value of the sectional image whose probability of including the background with the standard deviation of the prescribed characteristic value in other sectional images; and a background only sectional image statistic storage means stores the mean value and the standard deviation of the prescribed characteristic value of a location of the sectional image including only the background and the sectional image concerned and outputs them.

According to a forty ninth aspect of the invention, there is provided a device for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold and standard deviation reference background only sectional image selection means outputs a specified number of sectional images whose absolute value of skewness is less than the threshold value, in the order of having a nearer value to the standard deviation of the prescribed characteristic value from among sectional images, as the sectional image having a probability of including only the background, with the mean value and the standard deviation of the prescribed characteristic value of sectional images and the skewness of sectional images as inputs; a background only sectional image selection means outputs the sectional image having a standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is less than the threshold, and whose absolute value of skewness is less than the threshold, as the sectional image including only the background while comparing a standard deviation of the prescribed characteristic value of the sectional

CA 02249140 2001-06-05 image whose probability of including the background with the standard deviation of the prescribed characteristic value in other sectional images; and a background only sectional image statistic storage means stores the mean value and the standard deviation of the prescribed characteristic value of a location of the sectional image including only the background and the sectional image concerned and outputs them.

According to a fiftieth aspect of the invention, there is provided a device for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold and mean value and standard deviation reference background only sectional image selection means outputs a sectional image whose absolute value of skewness is less than the threshold value and whose mean value and standard deviation are of the nearest values to the mean value and standard deviation of the prescribed characteristic value from among sectional images, as the sectional image having a probability of including only the background, with the mean value and the standard deviation of the prescribed characteristic value of sectional images and the skewness of sectional images as inputs; a background only sectional image selection means outputs the sectional image having a standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is less than the threshold, and whose absolute value of skewness is less than the threshold, as the sectional image including only the background while comparing a standard deviation of the prescribed characteristic value of the sectional image whose probability of including the background with the standard deviation of the prescribed characteristic value in other sectional images; and a background only sectional image statistic storage means stores the mean value and the standard deviation of the prescribed characteristic value of a

CA 02249140 2001-06-05 location of the sectional image including only the background and the sectional image concerned and outputs them.

According to a fifty first aspect of the invention, there is provided a device for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold and mean value and standard deviation reference background only sectional image selection means outputs a sectional image whose absolute value of skewness is less than the threshold value and whose number is specified in order of nearer value of the mean value and the standard deviation of the prescribed characteristic value among sectional images as the sectional image whose probability of including only the background with the mean value and the standard deviation of the prescribed characteristic value of sectional images and the skewness of sectional images as inputs; a background only sectional image selection means outputs the sectional image having a standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is less than threshold, and whose absolute value of skewness is less than the threshold, as the sectional image including only the background while comparing the standard deviation of the prescribed characteristic value of the sectional image whose probability of including the background with the standard deviation of the prescribed characteristic value in other sectional images; and a background only sectional image statistic storage means for storing and outputting the mean value and the standard deviation of the prescribed characteristic value of a location of the sectional image including only the background and the sectional image.

According to a fifty second aspect of the invention, there is provided a device for object detection and background removal for enabling an object to be

CA 02249140 2001-06-05 automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold background only sectional image selection means for outputting a sectional image whose absolute value of skewness is less than the threshold, and whose probability of including only the background is of the highest value among sectional images as the sectional image whose probability of including only a background, with the mean value and the standard deviation of the prescribed characteristic value of sectional images and skewness of sectional images, and probability of including only the background in every sectional image within the area as inputs; a background only sectional image selection means for outputting the sectional image having the standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is less than threshold, and whose absolute value of skewness is less than the threshold, as the sectional image including only the background, while comparing the standard deviation of the prescribed characteristic value of the sectional image whose probability of including the background with the standard deviation of the prescribed characteristic value in other sectional images; and a background only sectional image statistic storage means for storing and outputting the mean value and the standard deviation of the prescribed characteristic value of a location of the sectional image including only the background and the sectional image.

According to a fifty third aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a skewness threshold background only sectional image selection means for outputting a sectional image whose absolute value of skewness is less than the threshold and whose probability of including only

CA 02249140 2001-06-05 background is of the highest value among sectional images, as the sectional image whose probability of including only a background, with the mean value, the standard deviation of the prescribed characteristic value of sectional images and skewness of sectional images, and probability of including only a background in every sectional image within the area as inputs; a background only sectional image selection means for outputting a sectional image having a standard deviation wherein the difference between the standard deviation thereof and the standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is less than the threshold and whose absolute value of skewness is less than the threshold, as the sectional image including only the background, while comparing a standard deviation of the prescribed characteristic value of the sectional image whose probability of including the background with the standard deviation of the prescribed characteristic value in other sectional images; and a background only sectional image statistic storage means for storing and outputting the mean value and the standard deviation of the prescribed characteristic value of a location of the sectional image including only the background and the sectional image.

According to a fifty fourth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means comprises a background exception sectional image selection means, and when a command for investigating sectional images including images with the exception of a background arrives, compares the mean value and the standard deviation of the sectional image including only the background and the mean value and the standard deviation of the prescribed characteristic value of estimated background of other sectional images, and if there exists a sectional image whose mean value and standard deviation of the prescribed characteristic value of the

CA 02249140 2001-06-05 background is not estimated, outputs the sectional image concerned, and if the mean value and the standard deviation of the prescribed characteristic value of the background in terms of all of the sectional images are estimated, issues a command to generate a threshold for the sake of object detection and background removal; a neighbourhood background only sectional image existence judgement means for comparing all of the mean values and standard deviations of the prescribed characteristic values both of sectional images including images with the exception of backgrounds, and sectional images including only background, located in the neighbourhood of the sectional images, and the mean values and standard deviations of the prescribed characteristic value of the estimated background of other sectional images, and when there exists only one sectional image having a mean value and standard deviation of the prescribed characteristic value in the neighbourhood, issues a command so as to estimate the mean value and the standard deviation of the prescribed characteristic value of the sectional image including images with the exception of the background, and when there exists no sectional image whose mean value and standard deviation of the prescribed characteristic value are estimated, issues a command so as to select the next sectional image; a mean value and standard deviation interpolation/extrapolation means, which when receiving a command for estimating a mean value and a standard deviation of the prescribed characteristic value in the sectional image including images with the exception of the background, estimates to be outputted, by averaging, the mean value and the standard deviation of the prescribed characteristic value of the sectional image including only the background in the neighbourhood thereof, and by averaging the mean value and the standard deviation of the prescribed characteristic value of the estimated background of the sectional image in the neighbourhood thereof, simultaneously outputs an estimated sectional image selection command signal so as to select the next sectional image; and

CA 02249140 2001-06-05 an estimated statistic storage means for storing to be outputted the mean value and the standard deviation of the prescribed characteristic value, thus selecting the sectional image whose probability of including only a background is high from sectional images involved within areas.

According to a fifty fifth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background sectional image selection means selects the sectional image whose probability of including only a background is high from sectional images involved within a plurality of areas.

According to a fifty sixth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background statistic estimation means comprises a center of gravity location calculation means for outputting a location of the center of gravity as a location of the center of gravity of an object of detection target, while comparing the mean value and the standard deviation of the prescribed characteristic value in sectional images including only the background, and calculating a location of the center of gravity with a location of sectional images without the mean value and the standard deviation of the prescribed characteristic value compared; a sectional image to center of gravity location distance calculation means for calculating to be outputted the distance between the location of center pixel of respective sectional images and the location of the center of gravity concerned, with the location of the center of gravity as the object of detection target; a background exception sectional image selection means, which when a command for investigating sectional images including images with the exception of a background arrives, compares the mean value and the standard deviation of the prescribed characteristic

CA 02249140 2001-06-05 value of the sectional image including only the background, and the mean value and the standard deviation of the prescribed characteristic value of the estimated background of other sectional images, and if there exists a sectional image whose mean value and standard deviation of the prescribed characteristic value of the background is not estimated, outputs the sectional image concerned, and if the mean value and the standard deviation of the prescribed characteristic value of the background in respect to all of the sectional images are instructed, issues a command so as to generate a threshold for the sake of object detection and background removal; a neighbourhood background only sectional image existence judgement means for investigating all of the mean values and standard deviations of the prescribed characteristic values both of sectional images including images with the exception of backgrounds and sectional images including only background located in the neighbourhood of the sectional images, and the mean values and standard deviations of the prescribed characteristic value of the estimated background of other sectional images, and when there exists only one sectional image whose mean value and standard deviation of the prescribed characteristic value is in the neighbourhood, issues a command so as to estimate the mean value and the standard deviation of the prescribed characteristic value of the sectional image including images with the exception of the background, and when there exists no sectional image whose mean value and standard deviation of the prescribed characteristic value are estimated, issues a command so as to select the next sectional image; a mean value and standard deviation interpolation/extrapolation means, which when receiving a command for estimating the mean value and the standard deviation of the prescribed characteristic value in the sectional image including images with the exception of the background, estimates to be outputted, by averaging, the mean value and the standard deviation of the prescribed characteristic value of the sectional image including only the background in the neighbourhood thereof, and by averaging the mean

CA 02249140 2001-06-05 value and the standard deviation of the prescribed characteristic value of the estimated background of the sectional image in the neighbourhood thereof, and simultaneously outputting an estimated sectional image selection command signal so as to select the next sectional image; and an estimated statistic storage means for storing to be outputted the mean value and the standard deviation of the prescribed characteristic value.

According to a fifty seventh aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background statisticestimation means comprises a background exception sectional image selection means, which when receiving a command for investigating sectional images including images with the exception of a background, compares the mean value and the standard deviation of the sectional image including only the background and the mean value and the standard deviation of the prescribed characteristic value of the estimated background of other sectional images, and if there exists a sectional image whose mean value and standard deviation of the prescribed characteristic value of the background is not estimated, outputs the sectional image concerned, and if the mean value and the standard deviation of the prescribed characteristic value of the background in terms of all of the sectional images are estimated, issues a command so as to generate a threshold for the sake of object detection and background removal; a neighbourhood background only sectional image existence judgement means for investigating all of mean values and standard deviations of the prescribed characteristic values both of sectional images including images with the exception of backgrounds and sectional images including only background located in the neighbourhood of the sectional images, and the mean values and standard deviations of the prescribed characteristic value of the estimated background of other sectional images, and when

CA 02249140 2001-10-23 there exists only one sectional image whose mean value and standard deviation of the prescribed characteristic value of the background is among the sectional images in a neighbourhood relationship, issues a command so as to estimate the mean value and the standard deviation of the prescribed characteristic value of the sectional image including images with the exception of the background, and when there exist no sectional image whose mean value and standard deviation of the prescribed characteristic value are estimated, issues a command so as to select the next sectional image; a mean value and standard deviation interpolation/extrapolation means, which when receiving a command for estimating the mean value and the standard deviation of the prescribed characteristic value in the sectional image including images with the exception of the background, estimates to be outputted a signal of the mean value and the standard deviation by averaging the mean value and the standard deviation of the prescribed characteristic value of the sectional image including only the background in the neighbourhood thereof, and by averaging the mean value and the standard deviation of the prescribed characteristic value of the estimated background of the sectional image in the neighbourhood thereof, and simultaneously outputs an estimated sectional image selection command signal so as to select the next sectional image; and an estimated statistic storage means for storing to be outputted the mean value and the standard deviation of the prescribed characteristic value estimated previously.

According to a fifty eighth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background statistic estimation means comprises a background exception pixel selection means for scanning a pixel successively in the order that commands are received to investigate the next pixel, and when the pixel which is scanned corresponds with the center pixel of the sectional image including only the background, causes the mean value and the

CA 02249140 2001-10-23 standard deviation of the prescribed characteristic value to be the statistic of the pixel in the sectional image including only the background, and when the pixel which is scanned does not correspond with the center pixel of the sectional image including only the background, outputs the location of the pixel, and subsequently, in cases where the scanning is completed in terms of all of the pixels, a command is issued so as to generate a threshold for the sake of object detection and background removal; a background exception pixel distance calculation means calculates to be outputted the distance between the location of the pixel which is scanned and the location of the center pixel of all of the sectional images including only the background; a mean value and standard deviation interpolation/extrapolation means implements estimation of the mean value and the standard deviation of the prescribed characteristic value of the background in the scanned location of the pixel, whereby the mean value and standard deviation of the prescribed characteristic value of the sectional image including only the background are weighted to average in response to the location of the center pixel and scanned location of pixel and the corresponding distance, and issues a command so as to select the next pixel; and an estimated statistic storage means for storing and outputting the mean value and the standard deviation of the prescribed characteristic value of the estimated background, and the mean value and the standard deviation of the prescribed characteristic value in the location of the center pixel of the sectional image including only the background.

According to a fifty ninth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background statistic estimation means comprises a center of gravity location calculation means investigates the mean value and the standard deviation of the prescribed characteristic value in the sectional images including only a background, and subsequently,

CA 02249140 2001-10-23 investigates a location of the sectional image having no mean value and no standard deviation of the prescribed characteristic value to calculate the location of the center of gravity, and outputs it by way of a location of the center of gravity of an object of detection target; a background exception pixel selection means scans a pixel successively in the order that commands are received for investigating the next pixel, and when the pixel which is scanned corresponds with the center pixel of the sectional image including only the background, causes the mean value and the standard deviation of the prescribed characteristic value to be the statistic of the pixel in the sectional image including only the background, and when the pixel which is scanned does not correspond with the center pixel of the sectional image including only the background, outputs the location of the pixel, and subsequently, in cases where the scanning is completed in terms of ail of the pixels, issues a command so as to generate a threshold for the sake of object detection and background removal; a background only sectional image center pixel selection means assumes a straight line connects the watched pixel and the location of the center of gravity of the object, and that a half straight line located at opposite sides of the location of the center of gravity of the object from the pixel on the straight line, and subsequently, selects whole center pixels of the sectional image including only the background, wherein sectional image intersected location is located on the half straight line while dropping a perpendicular to the straight line from the center pixel of the sectional image including only the background; a background exception pixel distance calculation means calculates to be outputted the distance between the scanned location of the pixel and the location of center pixel of the selected sectional image including only the background; a mean value and standard deviation interpolation/extrapolation means implements estimation of the mean value and the standard deviation of the prescribed characteristic value of the background in the scanned location of the pixel, whereby the mean value and standard deviation of the

CA 02249140 2001-10-23 prescribed characteristic value of the sectional image including only the background are weighted to average in response to the location of the center pixel and scanned location of the pixel and the corresponding distance, and issues a command so as to select the next pixel; and an estimated statistic storage means for storing and outputting the mean value and the standard deviation of the prescribed characteristic value of the estimated background, and the mean value and the standard deviation of the prescribed characteristic value in the location of the center pixel of the sectional image including only the background.

According to a sixtieth aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the background statistic estimation means does not possess a center of gravity location calculation means but the location of the center of gravity of an object is given beforehand.

According to a sixty first aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the threshold generation object detection and background removal means comprises a threshold generation means, which when receiving a command for calculating the threshold after the mean value and the standard deviation of the prescribed characteristic value of the background in all of the partial images has been estimated, obtains a first threshold in such a way that it causes the standard deviation to be multiplied by a constant and subtracted from the mean value, by using the mean value and the standard deviation of the prescribed characteristic of the estimated background in all of the sectional images, for the sake of detecting an object and removing the background, and a second threshold is obtained in such a way that it causes the standard deviation to be multiplied by a constant and added to the mean value, thus the first and the second threshold are

CA 02249140 2001-10-23 calculated over the whole picture area to be outputted, and a threshold processing means takes pixels involved between the two thresholds to be the background, and a background candidate area detection means for detecting an area which connects together these pixels determined to be the background, as a candidate area of the background; and a background judgement means takes the candidate area including the greatest number of sectional images whose probability of including only the background is high to be background areas, and taking other candidate areas to be target objects.

According to a sixty second aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the threshold generation object detection and background removal means comprises a threshold generation means, which when receiving a command for calculating the threshold after the mean value and the standard deviation of the prescribed characteristic value of the background in all of the partial images has been estimated, obtains a first threshold in such a way that it causes the standard deviation to be multiplied by a constant and subtracted from the mean value, by using the mean value and the standard deviation of the prescribed characteristic of the estimated background in all of the sectional images for the sake of detecting an object and removing the background, and a second threshold is obtained in such a way that it causes the standard deviation to be multiplied by a constant and added to the mean value, thus the first and the second threshold are calculated over the whole picture area to be outputted, and a threshold processing means takes pixels involved between the two thresholds to be the background; a background candidate area detection means for detecting an area which connects together these pixels determined to be the background, as a candidate area of the background, and a background judgement means takes the candidate area including

CA 02249140 2001-10-23 the greatest number of sectional images including only the background to be background areas, and taking other candidate areas to be target objects.

According to a sixty third aspect of the invention, there is provided an apparatus for object detection and background removal for enabling an object to be automatically detected minutely and accurately using contours, wherein the threshold generation object detection and background removal means comprises a threshold generation means, which when receiving a command for calculating the threshold after the mean value and the standard deviation of the prescribed characteristic value of the background in all of the partial images has been estimated, obtains a first threshold in such a way that it causes the standard deviation to be multiplied by a constant and subtracted from the mean value, by use ofthe mean value and the standard deviation of the prescribed characteristic of the estimated background in all of the sectional images for the sake of detecting an object and removing background, and a second threshold is obtained in such a way that it causes the standard deviation multiplied by a constant to be added to the mean value, thus the first and the second threshold are calculated over the whole picture area to be outputted, and a threshold processing means takes pixels involved between the two thresholds to be the background, a background candidate area detection means for detecting an area which connects together the pixels determined to be the background, as a candidate area of the background, and a background judgement means takes the candidate area including the smallest number of sectional images including images with the exception of backgrounds to be background areas and taking other candidate areas to be target objects.

The above and further objects and novel features of the invention will be more fully understood from the following detailed description when the same is read in connection with the accompanying drawings. It should be expressly understood,

CA 02249140 2001-06-05 however, that the drawings are for purpose of illustration only and are not intended as a definition of the limits of the invention.

Figure 1 is an explanation view showing one embodiment of an image of input image signal;

Figure 2 is an explanation view showing one example of a conventional object detection method;

Figure 3 is a block diagram showing a first embodiment of an object detection and background removal device of the present invention;

Figure 4 is an explanation view showing an example of division into 10 sectional image, the sectional image in the drawing is specified by row and column as

1-Aand so forth;

Figure 5 is an explanation view showing one example of processing result judging whether or not the sectional image is the background;

Figure 6 is an explanation view showing one example of processing for 15 estimating the background and the distribution thereof;

Figure 7 is a view explaining neighbourhood of the sectional image of processing for estimating the background and the distribution thereof;

Figure 8 is an explanation view showing one example of processing for estimating the background and the distribution thereof;

Figure 9 is an explanation view showing one example of processing for estimating the background and the distribution thereof;

Figure 10 is an explanation view showing one example of processing for estimating the background and the distribution thereof, and a center pixel location of the sectional image in the drawing is specified by row and column;

Figure 11 is a block diagram showing a second embodiment;

CA 02249140 2001-06-05

Figure 12 is a block diagram showing a constitution example of background sectional image selection means;

Figure 13 is an explanation view showing one example of processing for selecting a sectional image whose probability of including only the background is high;

Figure 14 is a block diagram showing a background statistic estimation means 314 of another configuration example of a background statistic estimation means 303;

Figure 15 is a block diagram showing a background statistic estimation means 315 of another configuration example of a background statistic estimation means 303;

Figure 16 is a block diagram showing a background statistic estimation means 316 of another configuration example of a background statistic estimation means

303;

Figure 17 is a block diagram showing one embodiment of threshold generation object detection and background removal means;

Figure 18 is a block diagram showing a configuration of a third embodiment of the object detection and background removal device;

Figure 19 is a block diagram showing a configuration of a fourth embodiment of the object detection and background removal device; and

Figure 20 is a view explaining edge detection processing.

A preferred embodiment of the method and apparatus for object detection and background removal and storage media for storing a program thereof according to the present invention will be described in detail referring to the accompanying drawings. Referring to Figures 3 to 20, there is shown the embodiment of the method and apparatus for object detection and background removal, and storage media for storing a program thereof of the present invention.

CA 02249140 2001-06-05

[FIRST EMBODIMENT]

Figure 3 is a block diagram showing configuration of a first embodiment of the object detection and background removal apparatus according to the present invention. For example, the input image signal 1 as shown in Figure 1 is inputted by way of an input image signal 200. The present object detection and background removal device 300 roughly comprises four means: a sectional image statistic calculation means 301, a background sectional image selection means 302, a background statistic estimation means 303, and a threshold generation object detection and background removal means 304.

The sectional image statistic calculation means 301 consists of respective configuration elements such as a sectional image division means 100, a mean value and standard deviation calculation means 101, and a sectional image statistic storage means 102. The sectional image division means 100 of these configuration elements inputs the input image signal 200, as shown in Figure 4, to divide the image into tile shaped sectional images, and outputs the sectional images 1-A, 2-A, 3-A in order, by way of a sectional image signal 201. The mean value and standard deviation calculation means 101 inputs the sectional image signal 201, to calculate a mean value and standard deviation of the brightness in each of the respective sectional images based on the respective equations (3) and (4), and then outputs them by way of sectional image statistic signal 202. The sectional image statistic storage means 102 stores the mean value of the brightness and the standard deviation of the respective sectional images, with the sectional image statistic signal 202 as the input, and outputs it by way of the sectional image statistic signal 203.

The background sectional image selection means 302 consists of respective configuration elements such as a minimum standard deviation criterion background only sectional image selection means 103, a background only sectional

CA 02249140 2001-06-05 image selection means 104, and a background only sectional image statistic storage means 105. The minimum standard deviation criterion background only sectional image selection means 103 inputs the sectional image statistic signal 203, judging the sectional image whose standard deviation of brightness is of the smallest value among those sectional images whose probability of including only the background is high, and then outputting this sectional image having a high probability background by way of the sectional image signal 204. The background only sectional image selection means 104 inputs both sectional image signal 204 and the sectional image statistic signal 203, thus comparing the standard deviation of the brightness of the sectional image whose probability of including only the background is high with the standard deviation of the brightness in another sectional image based on the equation (5). As a result of this comparison, it causes the background only sectional image signal 205 to be outputted, while regarding the sectional image having the standard deviation of the brightness within the threshold as the sectional image including only the background. The background only sectional image statistic storage means 105 inputs both the background only sectional image signal 205 and the sectional image statistic signal 203, and stores the mean value and the standard deviation of the brightness of the sectional image including only the background, and outputting it by way of the background only sectional image statistic signal 206.

The background statistic estimation means 303 consists of respective configuration elements such as a background-exception sectional image selection means 106, a neighbourhood background oniy sectional image existence judgement means 107, a mean value and standard deviation interpolation/extrapolation means 108, and a mean value and standard deviation storage means 109. In these configuration elements, the background-exception sectional image selection means 106 inputs the sectional image statistic signal 206, an estimation sectional image selection

CA 02249140 2001-06-05 command signal 211, and an estimation sectional image statistic signal 212, thus investigating both the background only sectional image statistic signal 206 and the estimation sectional image statistic signal 212 when the command to investigate the sectional image reaches thereto. In situations when the sectional image whose mean value and the standard deviation of the brightness byway of the background are not yet estimated exist, outputting of the sectional image by way of a background-exception sectional image signal, when the mean value and standard deviation of the brightness by way of the background are estimated with respect to the whole of the sectional images, outputs a threshold generation command signal 208 and generates a threshold for object detection and background removal. The neighbourhood background only sectional image existence judgement means 107 investigates a mean value and a standard deviation of the brightness of the sectional image located in the neighbourhood of the sectional image except the background, this being specified by the background-exception sectional image signal, with the background-exception sectional image signal 207, the background only sectional image statistic signal 206, and the estimation sectional image statistic signal 212 as the inputs.

In Figure 7, the watched sectional image is the sectional image 11, and the neighbourhoods are the sectional images located at the sectional images 12 to 19. In cases where when even one of the sectional images whose mean value and standard deviation of the brightness are estimated by way of the background exists in the neighbourhood, the outputting of a background statistic estimation command signal 209 estimates the mean value and the standard deviation of the sectional image specified by the background-exception sectional image signal 207, while in another case, the neighbourhood background only sectional image existence judgement means 107 outputs the estimated sectional image selection command signal 211 to select the next sectional image.

CA 02249140 2001-06-05

The mean value and standard deviation interpolation/extrapolation means 108 inputs the background statistic estimation command signal 209, the backgroundexception sectional image signal 207, the estimated sectional image statistic signal 212, and the background only sectional image statistic signal 206. In these inputs, when the mean value and standard deviation interpolation/extrapolation means 108 receives the command from the background statistic estimation command signal 209, it refers to the mean value and the standard deviation of the brightness by way of the background in the sectional image except the background specified by the background-exception sectional image signal 207. Further, the mean value and the standard deviation interpolation/extrapolation means 108 estimates the mean value and the standard deviation of the brightness byway of the background in the specified sectional image, by taking the average of the values shown in the equations (10), (11 ), and (12), while referring to the mean value and the standard deviation of the brightness of the sectional image estimated at the neighbourhood. The estimated mean value and standard deviation of the brightness is outputted by way of the estimated statistic signal 210, and simultaneously outputting the estimated sectional image selection command signal 211 so as to select the next sectional image. The estimated statistic storage means 109 inputs the estimated statistic signal 210, to store the mean value and the standard deviation of the brightness being estimated, and outputs it by way of the estimated sectional image statistic signal 212.

The threshold generation object detection and background removal means 304 consists of a threshold generation means 110 and a threshold processing means 111. In these configuration elements, the threshold generation means 110 inputs the threshold generation command signal 208, the estimation sectional image statistic signal 212, and the background only sectional image statistic signal 206. In these inputs, the mean value and the standard deviation of the brightness become complete

CA 02249140 2001-06-05 by way of the background in the whole of the sectional images by the threshold generation command signal 208. Consequently, when the command for calculating the threshold enters therein, the threshold generation object detection background removal means 304 calculates two thresholds over the whole of the picture by the equations (15), and (16) using the mean value and the standard deviation of the brightness in the whole sectional images, outputs it by way of the threshold signal 213. The threshold processing means 111 inputs the threshold signal 213 and the input image signal 200 to output an object detection background removal signal 214 such that the pixel which the equation (17) becomes true is the background, and another pixel which comes to be a detection target is the object, while comparing input image with two thresholds by the equation (17).

In the present embodiment, the sectional image division means 100, as shown in Figure 4, divides the image into tile-shaped sectional images. However, as shown in Figure 2, the present invention is also capable of being constituted such that the sectional images are arranged so as to overlap with each other, and the sectional images are disengaged to be arranged with each other.

In the present embodiment, the background sectional image selection means 302 selects the sectional image whose standard deviation of the brightness is of the most smallest value as the sectional image whose probability of including only the background is high. However, the present method includes numerous variations. For instance, the background sectional image selection means 302 can cause a specified number of the sectional images having the smaller value of the standard deviation of brightness to be selected or the background sectional image selection means 302 can also cause the sectional image having the nearest most value to the standard deviation of predirected brightness to be selected. The background sectional image selection means 302 can also cause the sectional image, or any specified number thereof, and

CA 02249140 2001-06-05 having a nearer number to the standard deviation of brightness to be selected. The background sectional image selection means 302 can also cause the sectional image having the nearest most value both to the mean value and the standard deviation of the brightness to be selected. The background sectional image selection means 302 can further cause the sectional image, or any specified number thereof having a nearer value to the mean value and the standard deviation of brightness to be selected. The background sectional image selection means 302 causes the predirected single or a plurality of sectional images to be selected. It is capable of constituting an effective invention in accordance with the target in any above described cases. Further, the sectional images which should become selection targets are selected from the sectional images included in an area instructed beforehand, or are selected from a plurality of the same areas as the instructed area respectively. It is also capable of constituting an effective invention in accordance with the target in any above described selection.

Furthermore, the neighbourhood background only sectional image existence judgement means 107 causes the sectional image which holds a side and top in common to be defined, for example, as in Figure 7, within the neighbourhood containing the sectional images. Somewhere else within the neighbourhood, there is defined the sectional image which only holds the side in common, or it is defined so that it causes the sectional image of the long distance to be included. These definitions enable effective invention to be constituted in accordance with the target.

[SECOND EMBODIMENT]

Figure 11 is a block diagram showing a configuration of a second embodiment of the object detection and background removal device according to the present invention. Supposing the input image signal 1 as shown in Figure 1 is inputted by way of an input image signal 200. The present object detection and background removal device 310 roughly comprises four means: a sectional image statistic

CA 02249140 2001-06-05 calculation means 311, a background sectional image selection means 312, a background statistic estimation means 303, and a threshold generation object detection and background removal means 304. Among them, the background statistic estimation means 303, and the threshold generation object detection and background removal means 304 are the same means as those of the first embodiment.

The sectional image statistic calculation means 311 consists of respective configuration elements such as a sectional Image division means 100, a mean value and standard deviation and skewness calculation means 120, and a sectional image statistic storage means 121. In these configuration elements, the sectional image division means 100 inputs the input image signal 200, as shown in Figure 4, to divide the image into tile shaped sectional images, thus outputting sectional images 1-A, 2-A,

3-A in order, by way of a sectional image signal 201. The mean value and standard deviation and skewness calculation means 120 inputs the sectional image signal 201, to calculate a mean value, a standard deviation, and a skewness of brightness in every respective sectional images based on respective equations (3), (4), and (7) thus outputting them by way of a sectional image statistic signal 220. The sectional image statistic storage means 121 inputs the sectional image statistic signal 220, and subsequently, storing the mean value, the standard deviation, and the skewness of the brightness of the respective sectional images, and outputs signals by way of the sectional image statistic signal 221.

The background sectional image selection means 312 consists of the respective constitution elements of the skewness threshold and minimum standard deviation background only sectional image selection means 122, the background only sectional image selection means 123, and the background only sectional image statistic storage means 105. In these constitution elements, the skewness threshold and minimum standard deviation background only sectional image selection means 122

CA 02249140 2001-10-23 inputs the sectional image statistic signal 221, thus regarding the sectional image whose skewness is determined to be less than the threshold selected beforehand based on the equation (8), and whose standard deviation of the brightness is the smallest value of the sectional images whose probability of including only the background is high. A high5 probability background only sectional image signal 204 and the sectional image statistic signal 221 of the background sectional image are inputted, and a comparison of the standard deviation of the brightness of the sectional image whose probability of including only background is high occurs, with the standard deviation of the brightness in other sectional images based on the equation (5). As a result of this comparison, the sectional image whose standard deviation of brightness is within the threshold and whose skewness is less than the threshold, is established by the equation (8). The result of this is taken to be the background only sectional image, that is, the sectional image including only the background. Further, the skewness threshold and minimum standard deviation background only sectional image selection means 122 inputs the background only sectional image signal 205 and the sectional image statistic signal 203, and subsequently, storing the mean value and the standard deviation of brightness of the sectional image including only the background, and outputs a signal by way of the background only sectional image statistic signal 206.

In the present embodiment, the background sectional image selection means 312 selects the sectional image whose absolute value of the skewness is less than the threshold and whose standard deviation of brightness is of the smallest value as the sectional image having a high probability of including only the background. However the present method includes the following variations. For instance, there is only one condition in which the absolute value of the skewness is less than the threshold. In addition to the above condition of the skewness, the background sectional image selection means 312 causes the sectional images, or a specified number thereof,

CA 02249140 2001-10-23 and which have a smaller value in relation to the standard deviation, to be selected. In addition to the above condition of the skewness, the background sectional image selection means 312 causes the sectional image having the nearest most value to the standard deviation of the predirected brightness to be selected. In addition to the above condition of the skewness, the background sectional image selection means 312 causes the sectional images which have a nearer value to that of the standard deviation of the predirected brightness to be selected. In addition to the above condition of skewness, the background sectional image selection means 312 causes the sectional image having the nearest most value both to the mean value and the standard deviation of the predirected brightness to be selected. In addition to the above condition of the skewness, the background sectional image selection means 312 can cause the sectional images having a nearer value to that of the mean value and the standard deviation of the brightness to be selected. In addition to the above condition of the skewness, the background sectional image selection means 312 can cause a single or a plurality of sectional images to be selected, it is capable of constituting an effective invention in accordance with the target in any above described cases.

Further, the sectional images which should become selection targets are selected from the sectional images included in an area, or are selected from a plurality of the same areas. It is also capable of constituting an effective invention in accordance with the target in any above described selection.

The respective sectional images which are included in the area have the probability of including only the background, thus the selected sectional image whose probability proves to be of maximum value, conforms to the conditions that the absolute value of skewness is less than the threshold. Where there is a plurality of the same areas, the invention is also capable of selecting sectional images from these

CA 02249140 2001-10-23 same areas. There will be described the ultimate example referring to Figure 12 hereinafter.

Figure 12 is a block diagram showing another configuration example of a background sectional image selection means 312. Another object detection background removal device is constituted in such a way that the present background sectional image selection means 313 is substituted for the background sectional image selection means 312 shown in Figure 11.

In Figure 12, the skewness threshold background only sectional image selection means 124 inputs both of the sectional image static signal 221, and the background only sectional image probability signal 222, namely, the skewness threshold background only sectional image selection means 124 inputs the mean value, the standard deviation and skewness and the probability of including only background which is supplied in every respective sectional images within a plurality of areas. Subsequently to these inputs, among the sectional images, the sectional image whose absolute value of the skewness is determined to be less than the threshold given beforehand according to the equation (8), and whose probability of including only the background is the highest, is selected from respective areas, and a signal is output by way of the sectional image whose probability of including only the background is high.

Figure 13 is an explanation view showing one example of processing a selected sectional image which has a high probability of including only the background, and which shows one example of the background only sectional image probability signal 222 in the shape of model. In the state of affairs where the detection object 3 is photographed at approximately center of the input image signal 1, the sectional image of the first area 25, the sectional image of the second area 26, the sectional image of the third area 27, and the sectional image of the fourth area which correspond to the four corners of the image, and the probability of including only each background is

CA 02249140 2001-06-05 extremely high. There is given the probability of including only the background beforehand in terms of the sectional image included in respective areas 25, 26, 27, or

28. In Figure 13, the probability 0.9 is given to the sectional image 1-A involved in the first area 25, the probability 0.7 is given to the sectional image 2-A, the probability 0.5 is given to 1 -B, the probability 0.3 is given to the sectional image 3-A, the probability 0.2 is given to the sectional image 2-B, and the probability 0.1 is given to the sectional image 1-C. Similarly, in terms of the second area 26, the respective probabilities 0.9, 0.7, 0.5, 0.3,0.2, and 0.1 are given to the respective sectional images 9-A, 8-A, 9-B, 7A, 8-B, and 9-C. In terms of the third area 27, the respective probabilities 0.9, 0.7, 0.5, 0.3,0.2, and 0.1 are given to the respective sectional images 1-1,2-1,1-H, 3-1,2-H, and 1-G. In terms of the fourth area 28, the respective probabilities 0.9, 0.7, 0.5, 0.3, 0.2, and 0.1 are given to the respective sectional images 9-I, 8-I, 9-H, 7-I, 8-H, and 9-G. The skewness threshold background only sectional image selection means 124 selects the sectional image whose probability of including only the background is highest among the sectional images which conform to the equation (8) in each respective area.

The background only sectional image selection means 123 inputs the high probability background only sectional image signal 204 and the sectional image statistic signal 221, and subsequently, comparing the standard deviation of the brightness of the sectional image whose probability of including only the background is high with that of the standard deviation of the brightness in other sectional images, based on the equation (5). As a result of this comparison, the background only sectional image selection means 123 outputs a signal by way of the background only sectional image signal 205 which is the sectional image including only background while judging the sectional image whose standard deviation of the brightness is within the threshold, and whose skewness is less than the threshold according to the equation (8). The background only sectional image statistic storage means 105 inputs both the

CA 02249140 2001-10-23 ⁹⁰ background only sectional image signal 205 and the sectional image statistic signal 203, and subsequently, stores the mean value and the standard deviation of the brightness of the sectional image which has been determined to have only the background involved, and outputs a signal by way of the background only sectional image statistic signal 206.

In the above-described the object detection and background removal device of the first and the second embodiments, the background statistic estimation means 303 has many kinds of variations. When it causes the statistic of the background in some sectional images to be estimated, the mean value and the standard deviation of the brightness of sectional images in the neighbourhood thereof are referred to. In the first and the second embodiments, the neighbourhood relationship in the whole image is defined by one, however, it is also capable of constituting an effective invention corresponding to the target due to the fact that it causes the sectional image being used for estimation purposes to be further limited from other neighbourhood sectional images based on the location of the center of gravity of the sectional image judged that image except the background is involved. Further, it is appropriate that the neighbourhood relationship is defined in every sectional image. There will be described hereinafter in terms of the former embodiment referring to Figure 14.

[EMBODIMENT OF BACKGROUND STATISTIC ESTIMATION MEANS]

Figure 14 is a block diagram showing a background statistic estimation means 314 of another configuration example of the background statistic estimation means 303 in the present invention. Another object detection and background removal device is constituted in such a way that the present background statistic estimation means 314 is substituted for the background statistic estimation means 303 shown in

Figures 3 or 11.

CA 02249140 2001-06-05

In Figure 14, a center of gravity location calculation means 125 inputs the background only sectional image statistic signal 206, firstly, and investigates the mean value and the standard deviation of the brightness of the sectional image which includes only the background. Secondly, investigating the whole location of the sectional image whose mean value and the standard deviation of the brightness do not exist, and thirdly, obtaining the location of the center of gravity of the object which is to be the detection target while calculating the location of the center of gravity, and outputting a signal by way of the center of gravity location signal 223. A sectional image-center of gravity distance calculation means 126 calculates distance between the location of the center pixel of whole sectional images and the location of the center of gravity of the object to be the detection target, with the center of gravity location signal 223 as the input, and outputs a signal by way of a sectional image-center of gravity location distance signal 224. A distance storage means 127 inputs the sectional image-center of gravity location distance signal 224, and subsequently, storing the distance between the whole sectional images and the location of the center of gravity, and outputs the signals by way of a sectional image-center of gravity location distance signal 225.

The background-exception sectional image selection means 106 inputs the background only sectional image statistic signal 206, and estimated sectional image selection command signal 211, and an estimated sectional image statistic signal 212, and investigates both the background only sectional image statistic signal 206 and the estimated sectional image statistic signal 212 occurs when the command to investigate these sectional images by the estimated sectional image selection command signal 211 enters the background exception sectional image selection means 106. When there exists the sectional image whose mean value and standard deviation of the brightness by way of the background are not yet estimated, outputting this sectional image by way of the background-exception sectional image signal 207, when the estimation of the

CA 02249140 2001-10-23 mean value and the standard deviation of the brightness by way of the background in terms of the whole of the sectional images is completed, a threshold generation command signal 208 is outputted so as to generate the threshold for the object detection and background removal.

The neighbourhood background only sectional image existence judgment means 107 inputs the background-exception sectional image signal 207, the background only sectional image statistic signal 206, the estimated sectional image statistic signal 212, and the sectional image-center of gravity location distance signal 225, in the sectional image which includes the image except the background specified by the background exception sectional image signal, and investigates the mean value and the standard deviation of the brightness in the sectional image within the neighbourhood whose distance from the center of gravity is farther than the sectional image. The scanned image is the sectional image 11 in Figure 8. When there exists a sectional image whose mean value and standard deviation of brightness estimated by one sectional image at the neighbourhood, a background statistic estimation command signal 209 is outputted to estimate the mean value and the standard deviation of the brightness of the sectional image specified by the background exception sectional image signal 207. In another case, outputting of the estimated sectional image selection command signal 211 selects the next sectional image.

The mean value and standard deviation interpolation/extrapolation means 108 receives the command by the background statistic estimation command signal 209, the background-exception sectional image signal 207, and the estimated sectional image statistic signal 212, thus obtaining the mean value shown in the equations (10), (11), and (12), while referring to the mean value and the standard deviation of the brightness of the sectional image except the background specified by the backgroundexception sectional image signal 207 to the background statistic estimation command

CA 02249140 2001-06-05 signal 209, while referring to the mean value and the standard deviation of the brightness of the sectional image including only the background of the neighbourhood to the background only sectional image statistic signal 206, and while referring to the mean value and the standard deviation of the brightness of the estimated sectional image of the neighbourhood to the estimated sectional image statistic signal 212. On account of these matters, it causes the mean value and the standard deviation of the brightness to be estimated byway of the background of the estimated sectional images. The mean value and the standard deviation of the estimated brightness are outputted by way of the estimation statistic signal 210, and simultaneously, it causes the estimated sectional image selection command signal 211 to be outputted so as to select next sectional image. The estimated statistic storage means 109 inputs the estimated statistic signal 210, and subsequently, storing the mean value and the standard deviation of the brightness by way of the estimated background, thus outputting the signal by way of the estimated sectional image statistic signal 212.

In the above-described matters, the mean value and the standard deviation of the brightness in every sectional image byway of the background statistic estimation means is estimated, however, it is also capable of estimating the mean value and the standard deviation in every pixel. Further, similar to the prior embodiment, it is capable of selecting information in use for estimation based on the location of center of gravity. These two embodiments will be described referring to Figures 15 and 16.

Figure 15 is a block diagram showing a background statistic estimation means 315, which is another configuration example of the background statistic estimation means 303 in the present invention. A different configuration of the object detection and background removal device is constituted in such a way that the present background statistic estimation means 315 is substituted for the background statistic estimation means 303 shown in Figures 3 or 11.

CA 02249140 2001-06-05

In Figure 15, the background exception pixel selection means 129 inputs the background only sectional image statistic signal 206 and the estimated pixel selection command signal 229. The background exception pixel selection means 129 causes each pixel to be scanned successively in order for a command for investigating the next pixel, in response to the estimated pixel selection command signal 229, to be received. The background-exception pixel selection means 129 investigates the background only sectional image statistic signal 206, and when the pixel which is scanned corresponds with the center pixel of the sectional image which includes only the background, causes the mean value and the standard deviation of the brightness to be the statistic ofthe pixel in the sectional image which includes only the background, and outputs a signal by way of a background only sectional image center pixel statistic signal 230. When the pixel which is scanned is different from the center pixel of the sectional image which includes only the background, the location of the pixel by way of the background exception picture element signal 226 is output. In cases where it causes the scanning to be completed in terms of the whole pixels, the backgroundexception pixel selection means 129 outputs a threshold generation command signal 208 so as to generate the threshold for object detection and background removal.

A background-exception pixel distance calculation means 130 calculates the distance between the location of the pixel which is scanned and the location of center of pixel ofthe sectional image which includes only whole backgrounds based on the equation (13), with the background only sectional image statistic signal 206 and the background-exception pixel signal 226, and outputs a signal by way of a pixel distance signal 227. The mean value and standard deviation interpolation/extrapolation means 131 inputs the background-exception pixel signal 226, the background only sectional image statistic signal 206, and the pixel distance signal 227, and subsequently, estimating the mean value and the standard deviation of the brightness by way of the

CA 02249140 2001-10-23 background in the scanned location of the pixel based on the equation (14), thus outputting the signal by way of an estimated statistic signal 228. When the estimation is completed, the mean value and standard deviation interpolation/extrapolation means 131 outputs an estimated pixel selection command signal 229 which issues a command to select the next pixel.

An estimated statistic storage means 132 inputs a background only sectional image center pixel statistic signal 230 and the estimated statistic signal 228, and subsequently, stores the mean value and the standard deviation of the brightness over the whole of the images, and outputs a signal by way of the estimated sectional image signal 212.

Figure 16 is a block diagram showing a background statistic estimation means 316 which is another configuration example of the background statistic estimation means 303 in the present invention. A different configuration of the object detection background removal device is constituted in such a way that the present background statistic estimation means 316 is substituted for the background statistic estimation means 303 shown in Figures 3 or 11.

A center of gravity location calculation means 125 inputs the background only sectional image statistic signal 206, and firstly, investigates the mean value and the standard deviation of the brightness in the sectional image which includes only the background. Secondly, investigates the entire location of the sectional image where the mean value and the standard deviation of the brightness do not exist occurs and, thirdly, obtaining the location of center of gravity of the object to be detection target while calculating the location of center of gravity, in order to output a signai by way of a center of gravity location signal 223.

A background-exception pixel selection means 129 inputs the background only sectional image statistic signal 206 and an estimated pixel selection command

CA 02249140 2001-10-23 signal 229 thereto. The background exception pixel selection means 129 causes each pixel to be scanned successively in order for a command for investigating the next pixel in response to the estimated pixel selection command signal 229 to be received. The background exception pixel selection means 129 investigates the background only sectional image statistic signal 206, and when the pixel which is scanned corresponds with the center pixel of the sectional image which includes only the background, causes the mean value and the standard deviation of the brightness to be the statistic of the pixel in the sectional image which includes only the background, and outputs a signal by way of a background only sectional image center pixel statistic signal 230. When the pixel which is scanned is different from the center pixel of the sectional image which includes only the background, outputting the location of the pixel occurs by way of the background exception picture element signal 226. In cases where it causes the scanning to be completed in terms of the whole pixels, the background exception pixel selection means 129 outputs a threshold generation command signal 208 so as to generate the threshold for object detection and background removal.

A background only sectional image center pixel selection means 133 inputs the background exception picture element signal 226, a center of gravity location signal 223, and the background only sectional image statistic signal 206, thus providing a straight line connecting the pixel which is scanned and the location of center of gravity of the object, and a half straight line located at an opposite side of the location of center of gravity of the object from that of the pixel on the straight line. The background only sectional image center pixel selection means 133 outputs the signal by way of a selected center pixel location signal 231, while selecting whole center pixels of the sectional image which includes only the background, wherein the sectional image intersected location is located on the half straight line while being perpendicular to the

CA 02249140 2001-10-23 straight line from the center pixel of the sectional image which includes only the background.

A background exception pixel distance calculation means 134 inputs the background only sectional image statistic signal 206, the background exception pixel signal 226, and a selected center pixel location signal 231, subsequently, calculating distance between the location of pixel which is scanned and the location of center picture element ofthe sectional image which includes only the selected background based on the equation (13), and outputs a signal byway of a pixel distance signal 227. A mean value and standard deviation interpolation/extrapolation means 135 inputs the background exception pixel signal 226, the background only sectional image statistic signal 206, the selected center pixel location signal 231, and the pixel distance signal 227, and subsequently, estimates the mean value and the standard deviation of the brightness by way of the background in the scanned location of the pixel from the statistic of the sectional image which includes only the selected background based on the equation (14), and outputs a signal by way of an estimated statistic signal 229. When the estimation is completed, outputting an estimated pixel selection command signal 229 occurs.

An estimated statistic storage means 132 inputs the background only sectional image center pixel statistic signal 230, and the estimated statistic signal 228, and subsequently, stores the mean value and the standard deviation ofthe brightness over the whole image, and outputs a signal by way ofthe estimated sectional image statistic signal 212.

Here, when the location of the center of gravity of the object of detection target is known beforehand, or the object of detection target is photographed at a location on the image, it is capable of being set in the situations where these locations are known beforehand while elim inating the center of gravity location calculation means.

CA 02249140 2001-10-23

As above, in the object detection and background removal device of the above first and second embodiment, the threshold generation object detection and background removal means 304 has numerous variations. A threshold generation object detection and background removal means 317 which is another configuration example of the threshold generation object detection and background removal means 304 will be described referring to Figure 17.

[MODIFIED EXAMPLE]

Figure 17 is a block diagram showing one embodiment of a threshold generation object detection and background removal means in the present invention. A different configuration of the object detection and background removal device is constituted in such a way that the present background statistic estimation means 317 is substituted for the background statistic estimation means 303 shown in Figures 3 or

11.

The threshold generation means 110 inputs the threshold generation command signal 208, the estimated sectional image statistic signal 212, and the background only sectional image statistic signal 206. Since the estimation of the mean value and the standard deviation of the brightness are completed by way of the background in the whole sectional images according to the threshold generation command signal 208, when the command of calculating the threshold is entered, comparing two thresholds of the equations (15) and (16) based on the equation (17) using the mean value and the standard deviation of the brightness in the whole sectional images, a background candidate pixel signal 232 is thus outputted, while regarding the pixel which the equation (17) comes to be true as the background.

A background candidate area detection means 137 inputs the background pixel signal 232, and subsequently, when detecting a group of the pixels linked with one another, outputs a background candidate area signal 233. A

CA 02249140 2001-06-05 background judgement means 138 inputs a high probability background only sectional image signal 204 and the background candidate area signal 233, thus outputting an objectdetection and background removal signal 214 while regarding an image including the largest number of sectional images whose probability of including only the background is as high as a background area, and while regarding another image as a target object.

With respect to the above background judgement means 138, it regards an image including the largest number of sectional images whose probability of including only the background is as high as a background area, however, by way of another example, it regards an image including the largest number of sectional images which include only the background as a background area, and it regards images which include the smallest number of sectional images except the background as a background area, so that it is capable of obtaining the same effect.

In the embodiments described-above, only the brightness is in use byway of information of the image, in another case, colour information or edge information instead of the brightness can be used or the colour information and the edge information can be arranged.

In general, colour image is constituted by three components of red green blue. In the above embodiments, only the brightness is in use, for instance, when large differences between the background and the target object are observed in terms of red components of the colour image, it is capable of constituting the invention agreed with the target object due to using oniy red components instead of brightness. Further, it is capable of treating the colour image consisting of three components of cyanogen magenta yellow being in use for printing or consisting of four components of cyanogen magenta yellow black similarly. Furthermore, it is capable of using them while combining respective components.

CA 02249140 2001-06-05

100

[THIRD EMBODIMENT]

An embodiment of the invention in which colour image consisting of three components of red green blue is inputted to be used together will be described referring to Figure 18.

Figure 18 is a block diagram showing a configuration of a third embodiment of an object detection and background removal of the present invention. The present object detection and background removal device 323 roughly consists of four means; a sectional image statistic calculation means 324, a background sectional image selection means 325, a background statistic estimation means 326, and a threshold generation object detection and background removal means 327.

The sectional image statistic calculation means 324 is constituted by following configuration elements. A sectional image division means 152 inputs an input image red signal 249, an input image green signal 250, and an input image blue signal 251, and subsequently, divides these three images of red green blue into tile shaped configurations of sectional images as shown in Figure 4 thus appending 1-A,

2-A, 3-A,.....to the respective sectional images in order, so that red component of the image is outputted by way of the sectional image red signal 252, green component of the image is outputted by way of the sectional image green signal 253, and blue component of the image is outputted by way of the sectional image blue signal 254. The sectional image statistic storage means 154 inputs a sectional image red statistic signal 255, a sectional image green statistic signal 256, and a sectional image blue statistic signal 257, and subsequently, storing the mean value, the standard deviation, and the skewness of each component of red green blue of the respective sectional images, and outputs a sectional image red statistic signal 258, a sectional image green statistic signal 259, and a sectional image blue statistic signal 260.

CA 02249140 2001-06-05

101

The background sectional image selection means 325 is constituted by following configuration elements. A skewness threshold background only sectional image selection means 155 inputs the sectional image red statistic signal 258, the sectional image green statistic signal 259, the sectional image blue statistic signal 260, and a background only sectional image probability signal 222, thus obtaining the sectional image of red green blue, having a skewness lower than the threshold according to the equation (8). Further, referring to the background only sectional image probability signal 222 representing probability including only the background in every sectional image shown in Figure 13, subsequently, selecting the sectional image whose probability is the most highest in the sectional images whose skewness is lower than the threshold from respective first area 25, second area 26, third area 27, and fourth area 28, thus regarding it as the sectional image with a high probability of including only the background, so that the sectional image is outputted by way of the high probability background only sectional image signal 204.

A background only sectional image selection means 156 inputs the high probability background only sectional image signal 204, the sectional image red statistic signal 258, the sectional image green statistic signal 259, and the sectional image blue statistic signal 260, subsequently, comparing the skewness of each component of red green blue of the sectional image with a high probability of including only the background with the threshold based on the equation (8), thus taking the sectional image whose respective standard deviation and skewness are within the threshold to be the sectional image which includes only the background, and subsequently, outputting the signal by way of a background only sectional image signal 205. A background only sectional image statistic storage means 157 inputs the background only sectional image signal 205, the sectional image red statistic signal 258, the sectional image green statistic signal 259, and the sectional image blue statistic signal

CA 02249140 2001-10-23

102

260, and subsequently, stores the mean value and the standard deviation of each component of red green blue in the sectional image deemed to include only the background, and outputs a background only sectional image red statistic signal 261, a background only sectional image green statistic signal 262, and a background only sectional image blue statistic signal 263.

The background statistic estimation means 321 is constituted by following configuration elements. A background-exception sectional image selection means 158 investigates both the background only sectional image red statistic signal 261 and the estimated sectional image statistic signal 264 when the command to investigate these sectional images is included by the estimated image selection command signal 211, with the background only sectional image red statistic signal 264 as the input. When there exists the sectional image whose mean value and standard deviation of red component by way of the background is not yet estimated, outputting the sectional image by way of the background exception sectional image signal 207 occurs. Further, when the mean value and the standard deviation of the red component by way of the background in terms of the whole sectional images are estimated, outputting the threshold generation command signal 208 occurs so as to generate the threshold for the sake of the object detection and background removal.

A neighbourhood background only sectional image existence judgement means 147 inputs the background-exception sectional image signal 207, the background only sectional image red statistic signal 261, and the estimated sectional image red statistic signal 264, thus investigating a mean value and a standard deviation of the red component in the sectional image located at the neighbourhood of the sectional image except the background specified by the background exception sectional image signal. It is assumed that a scanned sectional image is the sectional image 11 in Figure 7. Consequently, the neighbourhoods are the sectional images located at the

CA 02249140 2001-06-05

103 sectional images 12 to 19. Where there exists any one of the sectional images whose mean value and standard deviation of the brightness are estimated in the neighbourhood thereof, outputting the background statistic estimation command signal 209 so as to estimate a mean value and a standard deviation of each component of red green blue of the sectional image specified by the background exception sectional image signal 207 occurs. In a different case thereabove, the neighbourhood background only sectional image existence judgement means 147 outputs an estimated sectional image selection command signal 211 so as to select the next sectional image.

The mean value and standard deviation interpolation/extrapolation means 159 inputs the background statistic estimation command signal 209, the background exception sectional image signal 207, the estimated sectional image red statistic signal 264, the estimated sectional image green statistic signal 265, the estimated sectional image blue statistic signal 266, the background only sectional image red statistic signal 261, the background only sectional image green statistic signal 262, and the background only sectional image blue statistic signal 263. Subsequently, the mean value and standard deviation interpolation/extrapolation means 159 receives the command by the background statistic estimation command signal 209, referring to the mean value and the standard deviation of each component of red green blue of the background in the sectional image including image with the exception of the background specified by the background-exception sectional image signal 207 from the background only sectional image red statistic signal 261, the background only sectional image green statistic signal 262, and the background only sectional image blue statistic signal 263, and also referring to the mean value and the standard deviation of each component of red green blue of the background in the sectional image including only the background of the neighbourhood from the background only sectional image red statistic signal 261, the background only sectional image green statistic

CA 02249140 2001-06-05

104 signal 262, and the background only sectional image blue statistic signal 263. The mean value and standard deviation interpolation/extrapolation means 159 refers to the mean value and the standard deviation of each component of red green blue of the background in the estimated sectional image in the neighbourhood thereof from the estimated sectional image red statistic signal 264, the estimated sectional image green statistic signal 265, the estimated sectional image blue statistic signal 266, thus obtaining the mean value as shown in the equations (10), (11), and (12).

Due to the above-described matter, the mean value and the standard deviation of each component of red green blue of the background in the specified sectional image are outputted by way of the estimated sectional image red statistic signal 267, the estimated partial image green statistic signal 268, and the estimated sectional image blue statistic signal 269, and simultaneously, outputs the estimated sectional image selection command signal 211 so as to select the next sectional image. The estimated statistic storage means 149 takes an estimated sectional image red statistic signal 267, an estimated sectional image green statistic signal 268, and an estimated sectional image blue statistic signal 269 to be inputs, and subsequently, stores the mean value and the standard deviation of both the brightness and the edge which are estimated, and outputs signals by way of an estimated partial brightness image statistic signal 243 and an estimated partial edge image statistic signal 244.

The threshold generation object detection and background removal means 322 is constituted by the following configuration elements. A threshold generation means 161 inputs the threshold generation command signal 208, the estimated sectional image red statistic signal 264, the estimated sectional image green statistic signal 265, the estimated sectional image blue statistic signal 266, the background only sectional image red statistic signal 261, the background only sectional image green

CA 02249140 2001-06-05

105 statistic signal 262, and the background only sectional image blue statistic signal 263. By virtue of these inputs, the mean value and the standard deviation of each component of red green blue of the background in all of the sectional images become complete due to the threshold generation command signal 208. Consequently, when the command for calculating the threshold is received, calculation of two thresholds in every component covering all of the sectional images by using the mean value and the standard deviation of each component of red green blue in all of the sectional images occurs, according to the equations (15), and (16), and signals are output by way of a red threshold signal 270, a green threshold signal 271, and a blue threshold signal 272. A threshold processing means 162 inputs the red threshold signal 270, the green threshold signal 271, the blue threshold signal 272, the input image red signal 249, the input image green signal 250, and the input image blue signal 251. The threshold processing means 162 compares the red component of the inputted image with the two thresholds in terms of the red component according to the equation (17), similarly, this occurs in terms regarding the green component and the blue component, and which are also compared with the threshold, thus regarding the image which the equation (17) comes to be true in relation to each component of red green blue as the background, and regarding another image as the object to be the detection target, and an object detection background rejection signal 214 is outputted.

In the embodiment described above, red green blue is the colour equation of the input image utilized as it is, however it can be appropriate to convert to another colour equation. The examples of the colour equation are shown in the literatures: “Image Analysis Handbook” (Supervision of Mikio Takagi, Y. Shimoda, Tokyo University Publication Meeting, pp 485-491, 1991) in which HSI (H: hue, S: saturation, I: intensity) hexagonal pyramid colour model, or HSI bi-hexagonal pyramid colour model or the like is shown, and “New Edited Colour Science Hand Book (Edited

CA 02249140 2001-10-23

106 by Japan Colour Congress, Tokyo University Publication Meeting, pp 83-146, 1980) in which XYZ colour equation, Lab, luv, or the like is shown, it is capable of constituting an effective invention in compliance with the target object by the same configuration as the above-described embodiment in relation to any colour equation.

[FOURTH EMBODIMENT]

There will be described an embodiment of the invention which uses the brightness with the edge information referring to Figure 19.

Figure 19 is a block diagram showing a configuration of a fourth embodiment of an object detection and background removal device in the present invention. The present object detection and background removal device 318 roughly consists of four means: a sectional image statistic calculation means 319, a background sectional image selection means 320, a background statistic estimation means 321, and a threshold generation object detection and background removal means 322.

The sectional image statistic calculation means 319 is constituted by following configuration elements. An edge detection means 139 takes an input image signal 200 to be an input, thus calculating edge components in respective pixels to be an output signal by way of an edge image signal 234. Calculation of the edge component is described in the literature: “Image Analysis Hand Book (Supervision of Mikio Takagi, Y. Shimoda, Tokyo University Publication Meeting, pp 550-564, 1991). The present calculation of the edge component is based on one of Laplacian operator which is in use frequently.

In Figure 20, the edge component is defined in such a way that it causes the pixel 29 to be watched, subsequently obtaining the value added the brightness values of peripheral pixels 30, 31, 32, 33, 34, 35, 36, and 37, and then subtracting a value increased by eight times as many as the brightness value of the scanned pixel 29 therefrom. The edge image is capable of being obtained by implementing the edge

CA 02249140 2001-06-05

107 component calculation covering all of the pixels. A sectional image division means 140 inputs an input image signal 200, and an edge image signal 234, subsequently, dividing both images into tile shaped sectional images as shown in Figure 4 thus appending 1-A,

2-A, 3-A...... to the respective sectional images in order, so that the brightness component of the image is outputted by way of the partial brightness image signal 235 and the edge component of the image is outputted by way of the partial edge image signal 236. A mean value and standard deviation calculation means 141 inputs the partial brightness image signal 235 and the partial edge image signal 236, subsequently calculating the mean value and the standard deviation of the brightness and the edge in every sectional image based on the respective equations (3), and (4), thus outputting the signals by way of a partial brightness image statistic signal 237, and a partial edge image statistic signal 238. A sectional image statistic storage means 142 takes the partial brightness image statistic signal 237 and the partial edge image statistic signal 238 to be the input, and subsequently, storing the mean value and the standard deviation of brightness and edge of respective sectional images, thus outputting the signals by way of a partial brightness image statistic signal 239 and a partial edge image statistic signal 240.

A background sectional image selection means 320 is constituted by the following configuration elements. A minimum standard deviation reference background only sectional image selection means 143 takes the partial brightness image statistic signal 239 to be the input, and subsequently takes the sectional image whose standard deviation of brightness is the smallest value of those sectional images whose probability of including only the background is high, and outputs the sectional image by way of a high probability background only sectional image signal 204. A background only sectional image selection means 144 inputs the high probability background only sectional image signal 204, and subsequently, compares the standard deviation of the

CA 02249140 2001-10-23

108 brightness of the sectional image whose probability of including only the background is high with the standard deviation of the brightness in the other sectional images based on the equation (5), as well as comparing the standard deviation of the edge based on the equation (5), and thus takes the sectional image whose standard deviation of brightness and the edge are within the threshold, respectively, to be the sectional image which includes only the background, to be inputted by way of a background only sectional image signal 205. A background only sectional image statistic storage means 145 inputs the background only sectional image signal 205, the partial brightness image statistic signal 239, and the partial edge image statistic signal 240, and subsequently, storing the mean value and the standard deviation of the brightness and the edge in the sectional image deemed to include only the background, and outputting a background only partial brightness image statistic signal 241 and a background only partial edge image statistic signal 242.

A background statistic estimation means 321 is constituted by the following configuration elements. A background-exception sectional image selection means 146 inputs a background only partial brightness image statistic signal 241, the estimated sectional image selection command signal 211, and an estimated partial brightness image statistic signal 243, and investigates both the background only partial brightness image statistic signal 241 and the estimated partial brightness image statistic signal 243, when the command to investigate the sectional images, by virtue of the estimate sectional image selection command signal 211, is entered. Where there exists a sectional image whose mean value and standard deviation of the brightness by way of the background are not estimated yet, outputting of the sectional image by way of background-exception sectional image signal 207 occurs, while when the mean value and the standard deviation of the brightness in terms of whole sectional images are

CA 02249140 2001-10-23

109 estimated, outputs a threshold generation command signal 208 so as to generate a threshold for object detection and background removal.

A neighbourhood background only sectional image existence judgement means 147 inputs the background-exception sectional image signal 207, the background only partial brightness image statistic signal 241, and the estimated partial brightness image statistic signal 243, and investigates the mean value and the standard deviation of the brightness of the sectional image located in the neighbourhood of the sectional image except the background, specified by the background-exception sectional image signal 207. It is assumed that the scanned sectional image is the sectional image 11 in Figure 7, and that the neighbourhood is the sectional image located at the sectional images 12 to 19. When there exists one of the sectional images whose mean value and standard deviation of the brightness are estimated, outputting a background statistic estimation command signal 209 so as to estimate a mean value and a standard deviation of the brightness of the sectional image specified by the background exception sectional image signal 207 occurs, while where there exists another image thereof, outputting the estimated sectional image selection command signal 211 occurs, so as to select the next sectional image.

A mean value and standard deviation interpolation/extrapolation means 148 inputs the background statistic estimation command signal 209, the background20 exception sectional image signal 207, the estimated partial brightness image statistic signal 243, the estimated partial edge image statistic signal 244, the background only partial brightness image statistic signal 241, and the background only partial edge image statistic signal 242. By virtue of these inputs, when there is received a command, by virtue of the background statistic estimation command signal 209, causing the mean value to be obtained as shown in the equations (10), (11), and (12), while referring to the mean value and the standard deviation of the brightness of the background in the

CA 02249140 2001-10-23

110 sectional image except the background specified by the background-exception sectional image signal 207 from the background only partial brightness image statistic signal 241 and the background only partial edge image statistic signal 242, while referring to the mean value and the standard deviation of the brightness and the edge in the sectional image including only the background of the neighbourhood from the background only partial brightness image statistic signal 241 and the background only partial edge image statistic signal 242, while referring to the mean value and the standard deviation of the brightness and the edge in the estimated sectional image of the neighbourhood from the estimated partial brightness image statistic signal 243 and the estimated partial edge image statistic signal 244, and while referring to the mean value and the standard deviation of the brightness and the edge in the estimated partial brightness image statistic signal 243 from the estimated partial brightness image statistic signal 243 and the estimated partial edge image statistic signal 244. By virtue of these matters, it causes the mean value and the standard deviation of the brightness and the edge of the background in the sectional image being specified to be estimated. The estimated mean value and standard deviation of the brightness are then outputted by way of an estimated brightness statistic signal 245 and an estimated edge statistic signal 246, and simultaneously, outputting the estimated sectional image selection command signal 211 so as to select the next sectional image.

An estimated statistic storage means 149 takes the estimated brightness statistic signal 245 and the estimated edge statistic signal 246 to be inputs, and subsequently, storing the mean value and the standard deviation of the estimated brightness and edge, outputs an estimated partial brightness image statistic signal 243 and an estimated partial edge image statistic signal 244.

A threshold generation object detection and background removal means 322 is constituted by the following configuration elements. A threshold generation

CA 02249140 2001-10-23

111 means 150 takes the threshold generation command signal 208, the estimated partial brightness image statistic signal 243, the estimated partial edge image statistic signal 244, the background only partial brightness image statistic signal 241, and the background only partial edge image statistic signal 242 to be inputs, since the mean value and the standard deviation of the brightness and the edge are complete of the background of the whole sectional images in virtue of the threshold generation command signal 208, when the command for calculating the threshold is entered, calculating two thresholds covering the whole picture using the mean value and the standard deviation of the brightness and the edge in the whole sectional images in virtue of the equations (15) and (16), to output a brightness threshold signal 247 and an edge threshold signal 248. A threshold processing means 151 takes the brightness threshold signal 247, the edge threshold signal 248, the input image signal 200, and the edge image signal 234 to be the inputs, and subsequently, outputs an object detection and background removal signal 214 in terms of the fact that the image which the equation (17) holds to be true, in terms both of the brightness and the edge is the background, and that another image thereof is the object to be detection target, and while comparing the threshold in terms of the brightness with the brightness of the inputted image in virtue of the equation (17), and also similarly comparing the edge image with the threshold in terms of two edges.

Here, by way of the edge detection processing, there is described that one of the Laplacian Operator described in the literature: “Image Analysis Hand Book” (Supervision of Mikio Takagi, Y. Shimoda, Tokyo University Publication Meeting, pp 502-505, 1991) is in use. In another case, a different Laplacian Operator described in the same literature as above or Roberts Robinson Prewitt Kirsch or the like is in use, thereby it is capable of constituting the invention in accordance with the target.

Further, in the minimum standard reference background only sectional image selection

CA 02249140 2001-06-05

112 means 143, which selects the sectional image whose probability of including only the background is high because the standard deviation of the brightness is minimum, however, it is capable of constituting the invention in accordance with the target, by virtue of the fact that the standard deviation of the edge is in use and so forth.

As described above, according to the present invention, the method and apparatus of the object detection and background removal, and the storage media storing program thereof divides the location of the pixels of the input image constituted by the background and the object of the sectional images which include only the background, thus estimating the background on the input image based on the sectional image including the background. Subsequently, the estimated background is compared with the input image, thus the object of detection target is isolated. Namely, in the first place, the sectional image including only the background is selected while dividing the image into the sectional images, before estimating the background in the whole images based on the sectional image. On account of this procedure, it enables the background and the unknown object for instance, an object with brightness colour distribution, to be separated and detected accurately. In general, in order to separate accurately the background and the object to be the detection target, it is necessary to know background or distribution of characteristic value such as brightness, colour, edge and so forth which the object to be detection target is expressed on the image.

Consequently, according to the present invention, it enables the object with unknown brightness and colour distribution to be obtained accurately from the image having a virtually constant background as far as the boundary between the background and the object.

While preferred embodiments of the invention have been described using specific terms, such description is for illustrative purpose only, and it is to be understood

CA 02249140 2001-06-05

113 that changes and variations may be made without departing from the spirit or scope of the following claims.