Pattern noise correction for pseudo projections

ES 2 525 258 T3

DESCRIPTION

Pattern noise correction for pseudo-projections

Field of the invention

The present invention relates generally to the analysis of medical imaging data and more particularly to the correction of pattern noise in a biological cell imager.

Background of the invention

3D tomographic reconstructions require projection images as input. A projection image assumes that an object of interest is translucent to an exposure source such as a light source transmitted through the object of interest. The projection image, then, comprises an integration of absorption by the object along a ray from the source to the projection plane. Light in the visible spectrum is used as an exposure source in optical projection tomography.

In the case of the production of biological cell projections, cells are normally stained with hematoxylin, an absorbent dye that binds to proteins found on cell chromosomes. Cell nuclei are approximately 15 microns in diameter and, to promote reconstructions of subcellular elements, it is necessary to maintain submicron resolution. For submicron resolution, the wavelength of the illumination source is in the same spatial range as the biological objects of interest. This can result in undesirable refractive effects. As a result, a conventional projection image cannot be formed. To avoid these undesirable effects, as noted above, the camera aperture is kept open while the plane of focus is swept through the cell. This approach to imaging results in an equal sampling of the entire cell volume, resulting in a pseudo-projection image. A good example of an optical tomography system has been published as US Patent Application Publication 2004-0076319 and entitled "Method and Apparatus of Shadowgram Formation for Optical Tomography".

Johnathon R Walls et al., "Correction of artefacts in optical projection tomography", Physics in Medicine and Biology, IOP Publishing, vol. 50, no. October 19, 2005, reviews various sources of artifacts in optical CT images and proposes a background subtraction technique for correction.

Darrell A et al., "Noise reduction in fluorescence Optical Projection Tomography," IEEE International Workshop in Imaging Systems and Techniques, September 10-12, 2008, models observed noise in optical CT projections as a Gaussian model of mean zero dependent on intensity. Image averaging is proposed to reduce noise.

Pattern noise

Pattern noise represents a type of distortion that is fixed and present to the same degree for all pseudo-projection images acquired in any optical tomography system. The source of this distortion is any component in the optical path from illumination to image formation that causes the light to deviate from its ideal path in a way that is coherent from projection to projection. The pattern noise does not arise from the cell or any components on the cellular CT that are in motion during the collection of the pseudo-projection images.

Referring, for example, to Figure 2, a typical pseudo-projection image is shown showing some causes of pattern noise. These include dust and lighting variations. Also shown in figure 2 are two cells C1, C2 embedded in an optical gel. In a system that employs a CCD camera to acquire pseudo-projections or the like, sources of pattern noise include:

1. Not constant lighting,

two. Dust on a CCD camera,

3. Non-uniformity in the response of the CCD camera, and

Four. Illumination distortions arising as a result of dirt / debris on reflective surfaces found in the optical path.

Referring now to FIG. 2A, a selected portion 40 of the pseudo projection image is shown that has been enhanced as section 40A to better visually illustrate some subtle effects of pattern noise. Section 40A shows more subtle distortion resulting from dirt and debris on reflective surfaces in the optical path. This distortion is exemplified by taking a segment of the pseudo-projection and expanding it to fill the entire spatial gray-scale dynamic range. Note the speckled distortion in the background 44.

ES 2 525 258 T3

Distortions arising from pattern noise

Using an optical tomography system as described in Fauver, pseudo-projection images are formed as an object, such as a cell, is rotated. The pseudo-projection images formed are back-projected and intersected to form a 3D image of the cell. The pattern noise in the pseudo-projections also intersects and results in noise that is cumulative for the reconstruction of the object of interest. Although the noise in each pseudo-projection can be quite small, in the resulting reconstruction this noise can be quite large since the modeling can reinforce it constructively across multiple pseudo-projections.

Referring now to Figure 3, a reconstructed slide is shown that has been enhanced to show the effect of pattern noise on a reconstructed image. The swirling pattern 30 in the background is an obvious manifestation of the pattern noise.

Unfortunately, previously known techniques for spatial filtering do not adequately correct images since they do not effectively address the causes of pattern noise. Spatial filtering does not adequately correct for low frequency lighting variations. Furthermore, spatial filtering does not adequately remove impulse distortions, which arise from dust. Furthermore, the spatial frequency of speckle pattern noise is in the same range as other elements whose 3D reconstruction is desired. Consequently, a different approach is required for pattern noise removal.

The present invention described herein provides, for the first time, a novel and novel system and method for eliminating the detrimental effects of pattern noise in medical imagers.

Summary

This summary is provided to present a selection of concepts in simplified form that are further described in the Detailed Description below. This summary is not intended to identify key elements of the claimed issue, nor is it intended to be used to assist in determining the scope of the claimed issue.

One system and method for correcting projection images with pattern noise includes acquiring a set of projection images with an optical tomography system that includes a processor, where each of the set of projection images is acquired at a different viewing angle. A threshold is applied to each projection image to produce a set of threshold images. Each threshold image can optionally be dilated to produce a set of dilated images. The set of threshold images (or dilated images) are added together to form an overall image. Each of the threshold images (or dilated images) is processed to produce a set of binary images. The set of binary images are added together to form a set mask. The ensemble image is divided by the ensemble mask to give an image with background pattern noise. Each projection image is multiplied by a scale factor and divided by the background pattern noise to produce a ratio image that is filtered to produce a noise-corrected projection image.

Brief description of the drawings

Although the novel elements of the invention are set forth with particularity in the appended claims, the invention, both in terms of organization and content, will be better understood and appreciated, together with other objects and elements thereof, from the following description taken together with the drawings, in which:

Figure 1 is a highly schematic view of an optical projection tomography system that includes a standard noise correction processor.

Figure 2 shows a typical pseudo-projection image with pattern noise.

Figure 2A shows a selected portion of the pseudo-projection image of Figure 2 that has been enhanced to better visually illustrate some subtle effects of pattern noise.

Figure 3 shows a 3D reconstruction processed slice showing the effect of pattern noise.

Figure 4A shows a masked pseudo-projection of the cells shown in Figure 2 and Figure 4B shows a mask image for the cells.

Figure 5A shows a masked pseudo-projection of the cells shown in Figure 2 with capillary forward 45 ° and Figure 5B shows a mask image for the cells.

Figure 6 shows a masked pseudo-projection of the cells shown in Figure 2 with 45 ° inverted capillary and Figure 6B shows a mask image for the cells.

Figure 7 shows an image that results from the sum of all the masked pseudo-projections.

Figure 8 shows an image that results from the sum of all the mask images.

Figure 9 shows a grayscale noise image expanded to fill the dynamic range of the image.

Figure 10 shows a noise correction scheme.

Figure 11 illustrates the image of Figure 2 after noise correction application.

ES 2 525 258 T3

Figure 12A and Figure 12B show a comparison of image slices of a 3D reconstruction of pseudo-projections without noise correction and with noise correction, respectively.

Figure 13 shows a graphical representation of threshold selection criteria.

Description of the preferred embodiment

The following disclosure describes various embodiments and systems for generating images of an object of interest. Various elements of methods and systems in accordance with exemplary embodiments of the invention are set forth and described in the figures. It will be appreciated that methods and systems in accordance with other exemplary embodiments of the invention may include additional procedures or elements other than those shown in the figures.

Exemplary embodiments with respect to biological cells are described herein. However, it will be understood that these examples are for the purpose of illustrating the principles of the invention, and that the invention is not so limited. Additionally, methods and systems in accordance with various exemplary embodiments of the invention may not include all of the elements shown in these figures. Throughout the figures, like reference numerals refer to similar or identical components or procedures.

Unless the context requires otherwise, throughout the specification and claims that follow, the word "comprise" and variations thereof, such as, "comprises" and "comprising" are to be construed in an open, inclusive sense. which is like "including, but not limited to."

Reference throughout this specification to "an example" or "an exemplary embodiment", "an embodiment", or various combinations of these terms means that a particular element, structure, or feature described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, the occurrences of the phrase "in one embodiment" in various places throughout this specification are not all necessarily referring to the same embodiment. Furthermore, the particular elements, structures, or features can be combined in any suitable way in one or more embodiments.

Generally, as used herein, the following terms have the following meanings when used in the context of light microscopy processes:

"Capillary tube" has its generally accepted meaning and is intended to include transparent microcapillary tubes and equivalent elements with an internal diameter generally 500 microns or less.

"Depth of field" is the length along the optical axis within which the focal plane can shift before unacceptable image blurring occurs for a specified element. "Object" means a single cell, item, thing, particle, or other microscopic entity.

"Pseudo-projection" includes a single image representing a sampled volume of range greater than the native depth of field of a given set of optical elements. A concept of a pseudo-projection is taught in Fauver '744.

"Specimen" means a complete product obtained from a single test or procedure from an individual patient (eg, sputum submitted for analysis, a biopsy, or a nasal swab). A specimen can be composed of one or more objects. The result of the specimen diagnosis becomes part of the case diagnosis.

"Sample" means a finished cell preparation that is ready for analysis, including all or part of an aliquot or specimen.

As used in this specification, the terms "processor" and "computer processor" encompass a personal computer, a microcontroller, a microprocessor, an in situ programmable object array (FPOA), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), or any other motor, digital processing device or equivalent that includes related memory devices, streaming devices, indicating devices, input / output devices, displays, and the like.

Referring now to Figure 1, there is shown a highly schematic view of an optical projection tomography system that includes a standard noise correction processor. Cells 15 are suspended in a matching refractive index gel 12 contained in a capillary tube 18. Pressure 10 is applied to gel 12 to move cells into the optical path of a high-power microscope that includes an objective lens. 5. The objective lens 5 is swept or vibrated by, for example, a piezoelectric element (not shown). Capillary tube 18 is positioned to be swept by the vibrating objective lens 5. An illumination source 20 functions to illuminate objects, such as biological cells that pass through the field of view of the objective lens.

5. An image sensor 25 is positioned to acquire images transmitted from the objective lens 5. A plurality of pseudo-projection images, exemplified in this case by pseudo-projection images 22A, 22B, and 22C are acquired by image sensor 25 at viewing angles. variables as displayed by the rotating capillary tube 18. A noise-correcting image processor 35 is coupled to receive the pseudo-projection images. The corrected pseudo-projection images are then passed to a reconstruction processor 36 to produce 3-D images.

ES 2 525 258 T3

VisionGate, Inc. of Gig Harbor Washington, the assignee of this application, is developing an optical tomography system incorporating pattern noise correction under the trademark "Cell-CT ™". The Cell-CT ™ Optical Tomography System employs scores, designed to detect lung cancer in its pre-invasive and treatable phase. In an exemplary embodiment, the operation is as follows.

1. A specimen for examination is processed to remove non-diagnostic elements and is fixed and stained.

two. The specimen is then suspended in gel medium. Cells in a gel mix are then inserted into a glass microcapillary tube 18 of approximately 50 μ internal diameter 16.

3. Pressure is applied to the gel to move the cells into the optical path 14 of a high power microscope.

Four. Once the cells are in place, the tube is rotated to allow the capture of 500 high-resolution images of the desired object taken through the tube's 360 degrees of rotation. These images are projection image simulations created by integrating the light from the objective lens as the objective sweeps through the core. Simulated projection or pseudo-projection images therefore represent all nuclear content in a single image, taken from a single perspective.

5. Pseudo-projection images are processed to correct for residual noise and motion artifact.

6. The corrected pseudo-projections are processed using back-filtered projection to give a 3D tomographic representation of the cell. An exemplary section of such a 3D rendering is shown in Figure 3 for an Adenocarcinoma cell growing in culture.

7. Based on the tomographic reconstruction, elements that are used to detect cells with the characteristics of cancer and their precursors are computed. These elements are used in a classifier whose output designates the probability that the object under investigation is a cancer cell. The classifier outputs are based on a scoring system developed by VisionGate, Inc. called LuCED ™ scores.

Among other things, good quality reconstruction and classification depends on the good quality corrected pseudo-projections entered in the reconstruction algorithm in step 6. This document discloses a method to correct the pattern noise present in pseudo-projections at the time of capture of images. the data.

Pattern noise correction

As noted above, the pattern noise results from cumulative distortion. A pseudo-projection can be modeled as an ideal pseudo-projection plus the pattern noise. If the pattern noise is found, then the ideal non-noise pseudo-projection can be found by subtracting the pattern noise from the noisy pseudo-projection. Therefore, a challenge in performing subtractive correction is finding the image with pattern noise. The creation of an image with pattern noise is enabled by recognizing and using the fact that pseudo-projection images are composed of two image parts. A first image part is stable and common to the entire set of pseudo-projections and a second image part that is dynamic and changeable from one projection to the next. The dynamic part is the part that is associated with a sample such as a cell and other material that is suspended in the gel. In an optical tomography system design, the cell changes its position as the capillary tube is rotated. Since the cell and other material are dark relative to the background, the gel-suspended part of the image can be discarded by thresholding, leaving a partial representation of the stable part of the image.

Figure 4A shows an image after applying a threshold for the pseudo-projection of the figure.

two. Note that Figure 4B contains a mask image that is a binary version of the grayscale version of Figure 4A where all non-zero pixels are set to one. Figure 5A and Figure 5B and Figure 6A and Figure 6B show similar images for rotations plus and minus 45 degrees respectively from the position depicted in Figure 4A and Figure 4B. The axes are in pixel counts.

Referring now together to Figure 4A, Figure 5A, and Figure 6A, note that each image contains a different part of the background, or part of the image that contains pattern noise. In this observation is the key to the formation of the background image. The threshold images for the entire set of masked pseudo-projections can be added together to form an overall grayscale image as shown in Figure 7 for a full set of 500 pseudo-projections. It will be understood that, although a set of 500 pseudo-projections was used in some examples, the invention is not so limited and more or fewer pseudo-projections may be included in one set. The amount and speed of rotation can also be modified for different applications or results.

Referring now together to Figure 4B, Figure 5B and Figure 6B the mask images shown therein can be added together to form an overall mask. The summed images for a full set of 500 pseudo-projections are shown in Figure 8.

With reference now jointly and respectively to Figure 7 and Figure 8 it can be seen that nowhere in the images is there a point where certain background information is not available. By design, the background generally indicated as 70 and 70A in the respective figures is not substantially modulated across the

ES 2 525 258 T3 rotation of the tube. Cellular material is evidenced by modulated patterns, eg, 72 and 72A in the respective figures. Therefore, it is a good assumption that the background as computed by averaging the 500 pseudo-projections can be approximated by the background in any pseudo-projection. As a result, the pattern noise image can be found by dividing the ensemble grayscale image by the ensemble mask.

The result is shown in Figure 9 where the noisy image has been processed to expand the grayscale range to fill the entire dynamic range for the image. Note that Figure 9 shows that the noisy image represents all relevant distortions for which a correction is desired including

to. Illumination variation,

b. Powder, and

c. Mottled.

The correctness of any pseudo-projection is then a question of division.

Referring now to Figure 10 a noise correction scheme is shown. A typical 3D reconstruction for a biological cell requires the acquisition of 500 pseudo-projection images, PP0-PP499, each acquired as the capillary tube rotates through 500 incremental rotation angles, where PP0 is acquired at angle 0. ° and PP499 is acquired at approximately 360 °. In operation, loop 100 repeats through 500 incremental angles according to the order i = 0: 499. Each pseudo-projection, PPi, is processed through a threshold operation 104 to produce a threshold image. Optionally, the threshold image may then be stretched 106 to produce a stretched image. However, dilation is not an essential step for pattern noise correction and can be avoided or omitted. The dilated image or threshold image, as the case may be, is sent to an adder 110 that accumulates images with eliminated objects, and the sum of all the images forms an overall image 114. The dilated image or threshold image, as the case may be , is also processed to a binary image at 108 to form a mask which is added in the mask adder 112 finally producing a set mask 116. The thresholding procedures are further described below with reference to Figure 13. Thresholding, dilation, and masking operations can be implemented in a computer in the form of a computer program, dedicated processor, computer processor, electronic circuits, or the like including processors. and related devices listed above.

Referring now to FIG. 13, a graphical representation of a marked histogram with threshold selection criteria is shown. The correct operation of the noise correction algorithm depends on the correct selection of the threshold used to remove objects from pseudo-projections. In one example, threshold selection is accomplished through a two-part process and is done separately for each pseudo-projection. The two-part threshold selection process is based on two principles. First, a histogram 101 is generated that combines two influences of the image, the background and that of an object, such as a cell. Histogram 101 is characterized by a mode (Mode) and a maximum (Max). The mode represents the value that occurs most frequently, which in this case is the average value of the background. A cell in the image influences the histogram towards its dark side. Therefore, the variance in the background can be estimated by finding the difference between the maximum and the mode. An initial estimate for the threshold to separate the cell from the background in the image can therefore be made according to the formula: Threshold = 0.9 (2 * Mode - Max) as indicated by the dashed line 102 The estimated threshold is then applied to the image and the total area below the threshold is found.

The second principle that governs the calculation of the threshold stems from the fact that a profile of any one of several objects changes little from pseudo-projection to projection. This is because the capillary tube rotates in small increments from one pseudo-projection to the next. This fact is used to further fine-tune the threshold as it is iteratively adjusted until the total area of ​​pixels below the threshold is within 10% of the area for the previous threshold.

With reference again to the figure. 10, once the sums are available, the ensemble image 114 is divided by the ensemble mask to give the background pattern noise 118. Each PPi is multiplied by a scale factor (in this case, for example, 360000) and the product is divided by the background pattern noise 118. The ratio image is filtered by a low pass filter 122 that passes low frequency signals but attenuates signals with frequencies higher than the cutoff frequency, where the cutoff frequency is selected to filter out high frequency artifacts such as caused, for example, by camera noise. The cutoff frequency is selected to preserve the higher spatial frequencies for which a response is desired in the reconstruction. A filtered image is produced at 124 as a noise-corrected pseudo-projection.

Referring now to Figure 11, the correction result for the pseudo-projection of Figure 2 is shown. A comparison of Figure 11 with Figure 2 shows that the illumination variation has been corrected, the dust has been removed and the mottling has been substantially reduced.

ES 2 525 258 T3

Referring now to FIG. 12A and FIG. 12B, a comparison of image slices of a 3D reconstruction volume of pseudo-projections without noise correction and with noise correction respectively is shown. The first image in Figure 12A resulted from the reconstruction without noise correction. The second image in Figure 12B has been processed with noise correction. Note the much cleaner presentation of cellular detail for the noise-corrected reconstruction.

In an optical tomography system or similar system, noise correction according to the methods and systems described herein can be effectively performed when there is sufficient cell movement so that a background image can be generated in at least one small number of pseudo-projections. When this is not the case, noise correction may not be effective. In addition, the correct execution of the technique depends on the ability to eliminate the cells from the background, so that the gray matter in an image that results from the sum of all the masked pseudo-projections, as shown, for example, in the figure 7, picture only the background. This occurs when the algorithm that determines the threshold correctly identifies the threshold for segmenting cells. When thresholds are incorrectly identified, an image that results from the sum of all masked pseudo-projections can include cellular debris, leading to incorrect normalization. In this circumstance, the resulting pattern noise image, unlike the one shown in figure 9, shows a high variance. When the variance of the noisy image exceeds a predetermined value, noise correction cannot be performed effectively.

Although specific embodiments of the invention have been described and illustrated herein, it is recognized that numerous modifications and changes will occur to those skilled in the art. It should, therefore, be understood that the appended claims are intended to encompass all such modifications and changes that are within the scope of the invention.