System for detecting malagnancies

(71) Applicant(s)

Gary Rogers

Bell Meadow, Martham, GREAT YARMOUTH, Norfolk, NR29 4UA, United Kingdom

Tony Hedge

Burlingham House, Hewett Road, GREAT YARMOUTH, Norfolk. NR31 ONN, United Kingdom

Mark Shopland

Burlingham House, Hewett Road, GREAT YARMOUTH, Norfolk, NR31 0ΝΝ, United Kingdom (51) INTCL⁶

A61B5/00 (52) UK CL (Edition 0)

G1A AA1 AA20 AA3 AA6 AG 17 AHST AR6 (56) Documents Cited

US 4445516 A US 4428382 A US 4310003 A (58) Field of Search

UK CL (Edition 0 ) GIA AAJXAAMTAHCAHSL AHST,

H4F FAAJFGXX

INTCL⁶ A61B5/00

Online: WPI, INSPEC (74) Agent and/or Address for Service

Patrick Stone

Edenside Drive, ATTLEBOROUGH, Norfolk, NR17 2EL, United Kingdom (72) Inventor(s)

Gary Rogers (54) System for detecting malagnancies (57) A thermal imaging system comprising a thermal imaging device 10 which is able to detect local body temperature at a skin or sub-skin lesion and an adjacent point of normal skin with a resolution not less than 0.1*C. The thermogram produced may be processed by means of software filters in respect of texture, size of particles present, variations in intensity, spectral or spatial distribution, or density or intensity gradients in order to assist in distinguishing between benign and malignant lesions. The thermogram may be analysed with respect to time and a thermogram produced in response to stimulation by infra-red radiation. A neural network may also be used for analysis of the thermogram.

<img file="GB2311368A_D0001.tif" />

CD

<img file="GB2311368A_D0002.tif" />

<img file="GB2311368A_D0003.tif" />

<img file="GB2311368A_D0004.tif" />

At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy.

<img file="GB2311368A_D0005.tif" />

<img file="GB2311368A_D0006.tif" />

Fig. 1

<img file="GB2311368A_D0007.tif" />

<img file="GB2311368A_D0008.tif" />

Fig. 2

-1Svstem for Detecting Malignancies

This invention relates to a system for detecting skin and sub-skin malignancies, especially but not exclusively melanoma. Reference to sub-skin malignancies means malignancies which lie immediately beneath the skin, such as sub-skin growths of mole-like form. The invention is not intended to include detection of deep seated malignancies in respect of which the signal processing techniques hereinafter described would in general be inapplicable.

Melanoma is a highly malignant form of skin cancer which manifests itself as a mole-like lesion on the skin. However, at present, a cancerous lesion is not readily distinguishable from a benign mole without clinical examination at a hospital and after histological examination. Somewhat analogously, some forms of sub-skin breast cancer are not readily distinguishable from sub-surface cysts, and require removal and histological examination for proper identification.

Melanoma, particularly, is currently a major concern to the general public and referals of patients to hospitals by doctors in general practice have multiplied in number to such an extent that in many hospitals services are overburdened with clinical examinations for melanoma and analogous cancers to the detriment of other services which have to be carried out by the same personnel.

It would therefore be highly advantageous if a system was available for use by doctors in general practice and capable of reliably testing skin lesions to determine, at least in some cases, whether or not a particular lesion is malignant, thereby reducing the number of hospital referals.

According to the invention, there is provided a thermal imaging system comprising a thermal imaging device capable of detecting the local body temperature at a skin ο

-2or sub-skin lesion and at an adjacent point of normal skin with a resolution not less than 1/10th of a degree Centigrade, processing means for processing the output of the thermal imaging device to produce digital signals representative of the detected temperatures, a microprocessor programmed to analyse the temperature representative digital signals and develop video input signals having content indicative of the malignancy and/or non-malignancy of the lesion, and display means for receiving the video input signals and displaying said signals in an interpretable form. The thermal image or thermogram produced will initially be in a raw state. There is provided additional means for the software manipulation of this raw thermogram to produce transformed or processed images.

Thermal imagers (infra-red imagers) have already been used to perform thermography on deep seated malignancies but the end result has always been the obtaining of thermal profiles, that is temperature profiles. This invention allows temperature profiles to be taken in a quantitative and qualitative manner. Quantitative measures involve the measuring of certain parameters taken from the thermogram (or thermal image). Qualitative measures involve the assessment of the image after the image has been processed in some way, for instance by the use of software filters that transform the image data. Such processing, when suitable filters are used, can make lesions more apparent to the eye, that is qualitatively visible. The transformed images themselves can be assessed in a quantitative manner as well as a qualitative manner, that is the processed or transformed images can be subjected to precise measurement.

Thus, all matter emits infra-red radiation as a function of its absolute temperature, the infra-red energy being produced by molecular activity. As molecules differ so does the infra-red emitted, and in addition the intensity of emissions is dependent upon the number of molecules present. As cells are made up of a variety of different molecules, in varying numbers, a study of the infra-red spectrum, including intensity analysis, arising from a cell and the tissue it forms, ο

-3enables that cell or tissue to be adequately categorised, as it provides a record of the molecular activity of the cell or tissue. There are several methods of analysis by which a study of infra-red emissions proves fruitful in cancer detection, as follows:-

1. Total intensity of infra-red emitted, i.e. heat. By measuring the temperature profile of a lesion and comparing it with the corresponding profile of the normal surrounds it is sometimes possible to achieve a discemability that can indicate malignancy.

2. Spatial distribution analysis, referred to herein as texture analysis. Cancerous molecules may be grouped together intracellularly in a manner that can be characterised. Moreover cancerous cells may be grouped together in formations that can be categorised by a textural analysis.

3. Spectral distribution alaysis. A study of wavelength versus intensity can be achieved by measuring sub-bands of infra-red as they are emitted. If in a malignant tissue there are molecules that are actively emitting infra-red in a particular sub-band and not in another then this technique will allow them to be categorised.

4. Real time analysis. All tissue is living and therefore subject to rhythmicity. By analysing the variation of thermographic features with time it is possible to discern certain cancerous lesions from non-cancerous lesions.

5. Absorption analysis. Malignant cells have molecules present that absorb infra-red in a different manner to molecules present in non-malignant tissue. This study can be achieved by means of a real time analysis of lesions after stimulation with an external infra-red source. This technique can also be performed for differing sub-bands.

Spatial distribution analysis or textural analysis depends upon finding patterns within the raw thermogram, or areas of differing uniformity. To achieve this ο

-4software filters are used. These are computer programs that process the data held in the raw thermogram via a series of mathematical transformations. Such a process may involve a series of separate transformations in a particular algorithm. There are several different algorithms which can be used to good effect in transforming the raw thermogram so that the lesion is better seen qualitatively and/or others that produce transformed images that have features that are measurable. Textural analysis is the measurement or assessment of these transformed images.

Thus, in cancerous tissue, malignant cells are grouped together in different ways than are cells in non-malignant tissue. While this grouping may depend upon the results of a variety of factors, such an architecture lends itself to being detected by one or more of the thermographic techniques briefly described above.

A preferred thermal imaging device is a high resolution thermal imaging camera. One such camera utilises platinum silicide detectors and is thereby capable of detecting infra-red emissions, i.e. infra-red emissions from a suspect lesion and from normal skin. However, the thermal imaging device could alternatively be constituted by a CCD adapted to respond to infra-red wavelengths, or again by a thermal imaging camera utilising HgCdTe sensors.

The device may also include a conventional video camera for viewing a suspect lesion and the adjacent normal skin. The output of the video camera will preferably also be fed to the signal processing means to provide digital signals for input to the microprocessor, which may be programmed to correlate the visual image with the thermal image.

This makes it possible, for example, accurately to determine the size of a lesion and to manipulate the thermal image for ease of interpretation. For example, the thermal image of a suspect lesion may be subtracted from the thermal image of the ο

-5adjacent normal skin. It is also possible to produce thermal image maps that are representative of intensity levels of infra-red radiation emitted from a lesion both in one and two dimensions.

The thermal image obtained from a lesion can be analysed quantitatively in the following manner. The area of thermal influence of and around a lesion can be plotted in a three dimensional manner, with the area of the lesion being plotted on the x and y coordinates and the temperature at any point on the z coordinate, as shown in Figure 1 of the accompanying drawings. Such a plot enables temperature volume measurements to be taken. Temperature volumes are measured in cm x cm x degree Celsuis, and are a measure of the area of thermal influence a lesion has integrated with the temperature of all the points within that area. In the image shown in Figure 1, this temperature volume is the measure of the size of the crater seen (the depth of the crater is a measure of the relative temperature at any one point). The temperature volume can be used as a means of distinguishing the benign or malignant nature of a lesion.

Preferably, the thermal imaging device will be able electrically to pan, scroll and zoom over a suspect lesion, for example to select one or more particular areas of interest, as for cross-point temperature display.

Preferably, it will also be possible to display thermal images in real time, thereby digitally to capture transient thermal events, and also to make real time thermal subtractions enabling temperature comparisons.

The system may also include means for stimulating the suspect lesion to enable the response of the lesion to the stimulator to be measured. The stimulating means may, for example, include means for stimulating the lesion with infra-red radiation, preferably at differing wavelengths, to investigate the response of the lesion to absorption and re-emission of infra-red radiation. The stimulating means ο

-6may also comprise means for application of a low temperature stimulus, thereby for example to accentuate the temperature difference between the lesion and the adjacent normal skin.

Absolute temperature measurement, although possible, is not a prerequisite for practice of the invention. Most preferably, however, the system will be capable of measuring relative temperatures as between a substantial plurality of points of the surface of the subject at and around the suspect lesion. It will be appreciated that, by virtue of the use of a thermal imaging camera, the points can be as small as individual camera pixels, so that very accurate temperature comparison can be made, preferably at worst of the order of 1/1 Oth of a degree Centigrade.

It will be understood that although the foregoing description refers to lesions as skin lesions which are commonly associated with skin cancer, the system of the invention is also applicable to sub-skin cancers such as breast cancer and malignant lymph node involvement. Spectral distribution analysis enables detailed records to be built up to form a basis for comparisons thereby to enable lesions, whether of malignant or non-malignant nature, to be categorised by their spectral distribution characteristics.

A basic example of a detection system in accordance with the invention is shown in the accompanying drawing, the single figure of which shows the system diagrammatically.

In the drawing, a thermal imaging device 10, for example using platinum silicide detectors, provides an output to an image frame grabber board 12, which also receives image frame signals from a conventional video camera 14. Digital signals output from the grabber board 12 are fed to a microprocessor 16 programmed to produce signals which can be readily interpreted on a monitor screen 18. Optionally, an optical filter 20 can be placed in front of the thermal

-7imager 10 to enable selective detection of certain wavebands.

In use, the thermal imaging device 10, and likewise the video camera 14. electronically scan a suspect area of a subject, for example a skin lesion, to provide a display which represents, on the basis of differing amounts of infra-red radiation, the temperature differences between the surface of the lesion and the surface of the adjacent normal skin, with a minimum resolution of 1/100th of a degree Centigrade.

Figure 2 of the drawings shows the basic hardware of the detection system. However, also integral to the system is computer aided analysis software that enables the thermal image to be manipulated in a variety of ways. For instance, this software can enable mapping a two dimensional representation of the suspect lesion with a third dimension representing intensity of the thermal image. Allowance can be made for plotting additional data relating to spectral distribution onto this map or for enabling spatial distribution of the data, i.e. textural analysis. Frames grabbed at sequential time periods may be plotted in a manner that allows thermographic data to be plotted against time. Such data may relate to texture, spectral distribution or simple intensity levels, for example. The parameters of thermographic data that are collected by the thermal imager will also be capable of analysis in a variety of ways outlined below.

Textural analysis can be carried out in a variety of ways and relates to a study of the raw thermographic image after it has been processed by a variety of software filters. These show up the texture of the data held within the raw image in ways which produce further images interpretable by simple observational methods or by quantitative analysis. Quantitative analysis may measure the size of particles seen on a transformed image, or can involve the measurement of uniformity. Such patterns or texture produced by variations in intensity, spectral or spatial distribution, density, gradients of intensity can be measured and used as a means ο

-8of distinguishing benign from malignant lesions.

Spectral analysis can be effected by which means the wavelength of emitted infrared can be plotted against the intensity. This is achieved by modifying the thermal imager with filters interposed between imager and lesion, so filtering the infra-red energy before it reaches the sensing imager. By using a series of differing filters it will be possible to produce thermal images relating to differing sections of the infra-red spectrum; for instance images corresponding to 10 or 20 nanometre wide wavebands, although any size of waveband is possible.

Real time analysis of the thermal image can also be displayed. Rhythmicity in the infra-red image can be displayed, as can any variation in real time emissions from suspect lesions, and compared to surrounding tissue. A section of skin under study may be selected, either in one dimension or in two, and its behaviour over time recorded, with regard to intensity levels or intensity levels within selected wavebands or with regard to any parameter under study, such as textural analysis. Real time analysis may take the form of grabbing frames at sequential time periods, say for instance every 10 msec, or up to even 60 seconds.

The real time study of a lesion or a point on the lesion can be recorded as a data plot of activity over time, such activity' being intensity' of infra red emission or temperature. This time plot can be analysed in a variety of ways. Its pattern can be simply analysed quantitatively or it can be subjected to a variety of mathematical transformations. These transformations can be interpreted quantitatively or viewed qualitatively. Such transformations can be used to investigate the periodicity in the time plot, measure its spectral density function or spectral frequency for instance. The transformations can be used as a means of distinguishing benign from malignant lesions. Other techniques used would be to attempt to curve fit the time series or to estimate the randomness of the time series. Measures of the characteristics of such transformations can be used to c

-9distinguish benign from malignant lesions.

Real time analysis may occur in other circumstances in order to aid diagnosis of malignancy. With a suspect lesion under study an external infra-red source may be introduced and made to stimulate the suspect lesion for small periods of time.

The response of the lesion during and after stimulation will be noted by the thermal imager. In addition a cold stimulus may be introduced and the response of the lesion noted.

The ability of the thermal imager device to pan, scroll and zoom over a suspect lesion so as to concentrate on particular areas of interest makes it possible for different sections of the image to be analysed, i.e. certain areas of interest highlighted and compared to other areas, be it in one, two or three dimensions, or involving time as a fourth dimension or selected wavebands (sub-infra-red zones) as a fifth dimension.

There are a variety of ways in which textural analysis can be performed in addition to that of the use of software filters for producing transformations of the data. One such is the technique of looking at boundary sizes and boundray junctions. Such analysis is effected after boundaries have been made after setting criteria regarding parameters such as intensity levels, spectral distribution, or by correlating images with normal video camera images. Again, by means of techniques used in G1S (geographic imaging by satellite), the computer may be programmed to recognise certain features known to have associations with pathological entities, i.e. the texture of a melanoma is programmed into the computer so that when seen again the computer recognises it as such. Moreover, by means of mathematical modelling, the images analysed may be subjected to pattern recognition for fractal contents and analysis for fractal dimensions. Boundaries can be produced also on transformed images, such as to delineate the size of a feature seen in a transformed image. These boundaries can contain areas ο

-10that have certain features deemed of interest, such as an area of conformity with regard to texture, particle size etc. Thus the sizes of any areas of interest can be measured and used as a means of distinguishing benign from malignant.

The digital thermal images can also be analysed by applying chaos theory. A measure of the chaos in the system can be assessed, especially when images are taken in real time. Differences between benign and malignant lesions can be imaged by such analyses. To measure chaos in systems there are a variety of methods that can be employed: amongst these can be the following:-

a) measurement of the correlation dimension of the system

b) measurement of the Lyapunov exponent of the system

c) measurement of Poincare cross sectional maps

d) producing phase space diagrams of strange attractors

e) measurement of the fractal dimension

The computerised interpretation of the digital images is undertaken so as to measure these chaotic indicators for different areas of skin and for suspect lesions. These will then become clinicopathological indices. In addition, the strange attractors present in different clinicopathological states can be compared with the digital data obtained from the imagers, to see if a match can be made. The dimensions of the strange attractors measured can be varied according to the data obtained, but could for instance consist of data relating to real time collection of infra-red intensity levels at any given point or group of points together with spatial distribution data and spectral distribution data as time changes. Additionally there may be recorded rates of change of these parameters which can be utilised to produce chaotic interpretation of the data. Moreover, by measuring changes to the system (skin under study) as it is subjected to external influences such as active infra-red irradiation or cold stimulus it is possible to measure the chaotic parameters and record how they are altered, i.e. the measurements of chaos ο

-11(correlation dimension and Lyapunov exponents) can be measured not only in the resting state but also under stressed conditions (after application of external IR or cold stimuli).

From the static images taken pattern analysis can be made, for instance the spatial distribution of IR of a particular intensity can be analysed with regard to the patterns it produces. It is thus possible to model the patterns in a mathematical manner in order to understand the processes that produce them. Of particular interest is the analysis of any fractal qualities in the patterns detected. In this way a clinicopathological label can be given to areas under study, with the aim of enabling benign from disease state to be distinguished.

Of special interest is also the recognition of such measurements of variability as to whether they conform to the patterns associated with chaotic systems, or stable systems, or whether there is any evidence of bifurcations manifesting as bi- or quad-phasic steady state systems. These can then be labelled in a clinicopathological format. Evidence of period doubling behaviour may be evident in different stages of disease and by measuring the behaviour present, whether it be stable, bi-phasic or chaotic, with respect to whichever parameter is elected for measurement, the machine will be capable of establishing clinicopathological entities.

An additional manner in which the realtime data can be analysed is by training a neural network with the data obtained from image recordings and with a knowledge of the pathological outcome after histological analysis. The data fed into the neural network can be of a variety of types, both from static images in which textural differences will be foremost or from realtime data where the nonlinear streams of textural data will be witnessed. Textural in this sense can mean any measured infra-red parameter or grouping of parameters analysed for pattern differentiation, and can be analysed from static images or for images that are ο

-12constantly changing. For instance, the real time data stream may comprise intensity levels of infra-red, spatial distribution data, rate of change in intensity levels and in spatial distribution data, and spectral distribution data, all fed continuously into the neural network over a fixed time period. Without knowing the non-linear relationships that may exist between these and any other types of data inherent within the data stream the neural network will be able to learn how to recognise pathological states, and therefore predict disease states from the data obtained. However, the use of a neural network is not limited by the foregoing, Such a neural network can be employed in various ways as an interpretive tool.

In any particular system, not all the above-mentioned facilities for analysis may be present. However, it is essential to the invention that the microprocessor is able to output video signals having content which, by means of one or more forms of analysis, can enable malignant and non-malignant lesions, especially melanoma, to be distinguished.

The foregoing description sets out the basis for a test of malignancy in skin lesions

a) 3D plots temperature v area measuring temperature volumes

b) line profiles over lesions

c) transformations of the raw thermograms by the use of software filters

d) qualitative analysis of those lesions in c)

e) quantitative analysis of those lesions in c) by the following means :1. Analysis of thermal image thermograms

-132. Analysis of thermal time series

a) qualitative assessment

b) quantitative assessment

c) transformed time series and their assessment.

BAD ORIGINAL.

<img file="GB2311368A_D0009.tif" />