Untitled record
Abstract
In a touch-sensitive device, a panel directs signals, e.g., light, along actual detection lines extending across a surface area of the panel between pairs of coupling and decoupling points. Objects touching the surface area affect the light via frustrated total internal reflection (FTIR). A data processor processes (40) an output signal from a signal detector coupled to the decoupling points to generate a set of data elements indicative of detected energy for the actual detection lines. The set of data elements is further processed (42) to generate a set of matched elements indicative of estimated detected energy for fictitious detection lines having a position on the surface area that matches a standard geometry for tomographic reconstruction. The set of matched elements is further processed (44,46) by tomographic reconstruction to generate data indicative of a distribution of an energy-related parameter within the surface area.

Term
No projected expiry on record.
- Priority and filed
- Granted
- Today
23 claims: 12 independent, 11 dependent
- 1PATENTKRAV 1. Förfarande för möjliggörande av beröringsbestämning på basis av en utsignal från en pekkänslig apparat (100), varvid den pekkänsliga apparaten (100) innefattar en panel (4) som är utformad att leda signaler från ett flertal perifera inkopplingspunkter till ett flertal perifera utkopplingspunkter och därmed definiera faktiska detektionslinjer (D) som sträcker sig tvärs ett ytområde (1) på panelen (4) mellan par av inkopplingsoch utkopplingspunkter, minst en signalgenerator (2) som är kopplad till inkopplingspunktema för generering av signalerna, och minst en signaldetektor (3) som är kopplad till utkopplingspunktema för generering av utsignalen, varvid förfarandet innefattar:att bearbeta (40) utsignalen för generering av en uppsättning dataelement, varvid dataelementen indikerar detekterad energi för åtminstone en delmängd av de faktiska detektionslinjema (D), att bearbeta (42) uppsättningen dataelement för generering av en uppsättning matchade element, varvid de matchade elementen indikerar skattad detekterad energi för fiktiva detektionslinjer som har en position på ytområdet (1) som matchar en standardgeometri för tomografisk rekonstruktion, och att bearbeta (44,46) uppsättningen matchade element genom tomografisk rekonstruktion för generering av data som indikerar en fördelning av en energirelaterad parameter inom åtminstone en del av ytområdet (1).
- 2Förfarande enligt krav 1, varvid steget att bearbeta (40) utsignalen innefattar:att generera dataelementen i en tvådimensionell mätrymd, varvid vaije dataelement representerar en faktisk detektionslinje (D) och är definierat av ett signalvärde och två dimensionsvärden ( p, s;β, a) som bestämmer positionen för den faktiska detektionslinjen (D) på ytområdet (1).
- 3Förfarande enligt krav 2, varvid steget att bearbeta (42) uppsättningen dataelement innefattar:att generera skattade signalvärden för de matchade elementen vid förbestämda positioner i den tvådimensionella mätrymden, varvid de förbestämda positionerna motsvarar de fiktiva detektionslinjema.
- 4Förfarande enligt krav 3, varvid de skattade signalvärdena genereras genom interpolering baserat på signalvärdena för dataelementen.
- 5Förfarande enligt krav 4, varvid vaije skattat signal värde genereras genom interpolering av signalvärden för närliggande dataelement i den tvådimensionella mätrymden.
- 6Förfarande enligt krav 4 eller 5, varvid steget att bearbeta (42) uppsättningen dataelement vidare innefattar:att inhämta en förbestämd tvådimensionell 535 005 interpolationsfunktion (IF) med noder som motsvarar uppsättningen dataelement, och att beräkna de skattade signalvärdena enligt interpolationsfunktionen (IF) och baserat på dataelementens signalvärden.
- 7Förfarande enligt krav 6, vidare innefattande:att ta emot uteslutningsdata som identifierar ett eller flera dataelement som ska uteslutas, varvid steget att bearbeta (42) dataelementen innefattar att identifiera den nod som motsvarar vaije dataelement som ska uteslutas, omforma den förbestämda interpolationsfunktionen (IF) utan vaije sådan identifierad nod, och beräkna de skattade signalvärdena enligt den omformade interpolationsfunktionen (IF') och baserat på dataelementens signalvärden i den omformade interpolationsfunktionens (IF) noder.
- 8Förfarande enligt något av kraven 3-7, varvid de matchade elementen arrangeras i rader och/eller kolumner i den tvådimensionella mätrymden.
- 9Förfarande enligt krav 8, varvid de matchade elementen arrangeras med ekvidistanta avstånd inom var och en av nämnda rader och/eller kolumner.
- 10Förfarande enligt något av kraven 3-9, varvid steget att bearbeta (44,46) uppsättningen matchade element innefattar:att applicera (44) en endimensionell högpassfiltrering på de matchade elementen i den tvådimensionella mätrymden för att generera filtrerade element, och att bearbeta (46) de filtrerade elementen för att generera en uppsättning bakprojektionsvärden som indikerar nämnda fördelning.
- 11Förfarande enligt något av kraven 2-10, varvid ytområdet (1) definierar ett mätområde i den tvådimensionella mätrymden, och varvid steget att bearbeta uppsättningen mätdata innefattar, om de faktiska detektionslinjema (D) som ges av den geometriska placeringen av inkopplings- och utkopplingspunktema resulterar i åtminstone en sammanhängande region utan dataelement inom mätområdet:att inhämta en förbestämd uppsättning skattade mätpunkter inom den sammanhängande regionen, och att, för varje skattad mätpunkt, identifiera positionen för en motsvarande fiktiv detektionslinje på ytområdet;att identifiera, för vaije korsningspunkt mellan den motsvarande fiktiva detektionslinjen och de faktiska detektionslinjema (D) och/eller mellan den motsvarande fiktiva detektionslinjen och de fiktiva detektionslinjema för uppsättningen matchade element, ett korsningspunktsvärde som det minsta signalvärdet hos alla dataelement som motsvarar de med korsningspunkten associerade faktiska detektionslinjema (D);och att beräkna ett signalvärde för den skattade mätpunkten som funktion av korsningspunktvärdena.
- 12Förfarande enligt krav 11, varvid signalvärdet för den skattade mätpunkten ges av det största korsningspunktsvärdet. 535 005
- 13Förfarande enligt krav 11, vidare innefattande att, för vaije skattad mätpunkt:identifiera ett antal lokala maxima i korsningspunktsvärdena, och beräkna signalvärdet för den skattade mätpunkten som en kombination av nämnda lokala maxima.
- 14Förfarande enligt något av kraven 2-13, varvid dimensionsvärdena innefattar en vridningsvinkel för detektionslinjen i panelens (4) plan, och ett avstånd till detektionslinjen i panelens (4) plan från ett förbestämt origo.
- 15Förfarande enligt något av kraven 2-13, varvid dimensionsvärdena innefattar ett vinkelläge för detektionslinjens inkopplings- eller utkopplingspunkt, och en vridningsvinkel for detektionslinjen i panelens (4) plan.
- 16Förfarande enligt krav 15, varvid standardgeometrin är en solfjädergeometri, varvid pekytan (1) har en icke-cirkulär omkrets, och varvid vinkelläget definieras av en korsning mellan den faktiska detektionslinjen (D) och en fiktiv cirkel (C) som är anordnad att omringa pekytan (1).
- 17Förfarande enligt något av kraven 1-15, varvid standardgeometrin är en av en parallell geometri och en solfjädergeometri.
- 18Förfarande enligt något av föregående krav, varvid nämnda signaler innefattar en av elektrisk energi, ljus, magnetisk energi, ljudenergi och vibrationsenergi.
- 19Förfarande enligt något av föregående krav, varvid panelen (4) avgränsar en pekyta (1) och en motstående yta (5;6), varvid nämnda minst en signalgenerator (2) är anordnad att tillhandahålla ljus inuti panelen (4), på ett sådant sätt att ljuset propagerar från inkopplingspunktema medelst intemreflektion mellan pekytan (1) och den motstående ytan (5;6) till utkopplingspunktema för att detekteras av nämnda minst en signaldetektor (3), och varvid den pekkänsliga apparaten (100) är utformad på ett sådant sätt att det propagerande ljuset dämpas lokalt av ett eller flera pekytan (1) berörande föremål (7).
- 20Datorprogramprodukt innefattande programkod som, när den exekveras i ett databearbetningssystem, är anpassad att utföra förfarandet enligt något av kraven 1-19.
- 21Anordning som medger beröringsbestämning baserat på en utsignal från en pekkänslig apparat (100), varvid nämnda pekkänsliga apparat (100) innefattar en panel (4) som är utformad att leda signaler från ett flertal perifera inkopplingspunkter till ett flertal perifera utkopplingspunkter och därmed definiera faktiska detektionslinjer (D) som sträcker sig tvärs ett ytområde (1) på panelen (4) mellan par av inkopplings- och utkopplingspunkter, organ (2,12) för generering av signalerna vid inkopplingspunktema, och organ (3) för generering av utsignalen baserat på detekterade signaler vid utkopplingspunktema, varvid nämnda anordning innefattar:organ (400) för mottagning av utsignalen;535 005 organ (402) för bearbetning av utsignalen för generering av en uppsättning dataelement, varvid dataelementen indikerar detekterad energi för åtminstone en delmängd av de faktiska detektionslinjema (D);organ (404) för bearbetning av uppsättningen dataelement för generering av en uppsättning matchade element, varvid de matchade elementen indikerar skattad detekterad energi för fiktiva detektionslinjer som har en position på ytområdet (1) som matchar en standardgeometri för tomografisk rekonstruktion;och organ (406,408) för bearbetning av uppsättningen matchade element genom tomografisk rekonstruktion för generering av data som indikerar en fördelning av en energirelaterad parameter inom åtminstone en del av ytområdet (1).
- 22Pekkänslig apparat, innefattande:en panel (4) som är utformad att leda signaler från ett flertal perifera inkopplingspunkter till ett flertal perifera utkopplingspunkter och därmed definiera faktiska detektionslinjer (D) som sträcker sig tvärs ett ytområde (1) på panelen (4) mellan par av inkopplings- och inkopplingspunkter, organ (2,12) för generering av signalerna vid inkopplingspunktema;organ (3) för generering av en utsignal baserat på detekterade signaler vid utkopplingspunktema;och anordningen (10) som medger beröringsbestämning enligt krav 21.
- 23Pekkänslig apparat, innefattande:en panel (4) som är utformad att leda signaler från ett flertal perifera inkopplingspunkter till ett flertal perifera utkopplingspunkter och därmed definiera faktiska detektionslinjer (D) som sträcker sig tvärs ett ytområde (1) på panelen (4) mellan par av inkopplings- och utkopplingspunkter;minst en signalgenerator (2, 12) som är kopplad till inkopplingspunktema för att generera signalerna;minst en signaldetektor (3) som är kopplad till utkopplingspunktema för att generera en utsignal;och en signalprocessor (10) som är kopplad att ta emot utsignalen och är utformad att: bearbeta utsignalen för generering av en uppsättning dataelement, varvid dataelementen indikerar detekterad energi för åtminstone en delmängd av de faktiska detektionslinjema (D), bearbeta uppsättningen dataelement för generering av en uppsättning matchade element, varvid de matchade elementen indikerar skattad detekterad energi för fiktiva detektionslinjer som har en position på ytområdet (1) som matchar en standardgeometri för tomografisk rekonstruktion, och 535 005 bearbeta uppsättningen matchade element genom tomografisk rekonstruktion för generering av data som indikerar en fördelning av en energirelaterad parameter inom åtminstone en del av ytområdet (1). 535 005 1/25 2 3 Y A □8H 3®080^08 0® 2 O§ 3 CS
Independent claims23
274 paragraphs in 3 sections, as filed
(54) Title: Determination of contact by tomographic reconstruction (56) Published publications: - (47) Abstract:
In a touch-sensitive apparatus, a panel of signals, e.g., light, passes along actual detection lines extending across a surface area of the panel between pairs of switch-on and switch-off points. Objects that affect the surface area affect light via frustrated total internal reflection (FTIR). A data processor processes (40) an output of a signal detector coupled to the cut-off points to generate a set of data elements indicating detected energy for the actual detection lines. The set of data elements is further processed (42) to generate a set of matched elements indicating estimated detected energy for fictive detection lines having a surface area position that matches a standard tomographic reconstruction geometry. The set of matched elements is further processed (44,46) by tomographic reconstruction to generate data indicating a distribution of an energy-related parameter within the surface area.
<img file="SE535005C2_D0001.tif" />
Extract touch data
535 005
SUMMARY
In a touch-sensitive apparatus, a panel of signals, e.g., light, passes along actual detection lines extending across a surface area of the panel between pairs of switch-on and switch-off points. Objects that affect the surface area affect light via frustrated total internal reflection (FTIR). A data processor processes (40) an output of a signal detector coupled to the cut-off points to generate a set of data elements indicating detected energy for the actual detection lines. The set of data elements is further processed (42) to generate a set of matched elements indicating estimated detected energy for fictive detection lines having a surface area position that matches a standard tomographic reconstruction geometry. The set of matched elements is further processed (44,46) by tomographic reconstruction to generate data indicating a distribution of an energy-related parameter within the surface area.
535 005 i
DETERMINATION OF CONTACT THROUGH TOMOGRAPHIC RECONSTRUCTION
Technical area
The present invention relates to touch sensitive panels and data processing techniques in relation to such panels.
Background of the invention
Increasingly, touch-sensitive panels are used to enter data into computers, electronic measuring and test equipment, gaming devices etc. The panel can be provided with a graphical user interface (GUI), through which a user can interact with, for example, a pointer, a pen or one or more fingers. The user interface can be static or dynamic. For example, a static GUI may take the form of printed material placed over, under, or inside the panel. A dynamic GUI can be provided by a display screen integrated with or located below the panel, or be present as a projected image on the panel by a projector.
There are a number of known techniques for making the panel sensitive, for example, by using cameras for capturing light scattered from the touch points of the panel, or by inserting wiring harnesses, capacitive sensors, strain gauges, etc. into the panel.
US2004 / 0252091 describes an alternative technique based on frustrated total internal reflection (FTIR). Light layers are connected to the panel to propagate inside the panel via total internal reflection. When an object is brought into contact with a surface of the panel, two or more layers of light will be dimmed locally at the point of contact. Rows of light sensors are placed around the perimeter of the panel to detect the received light for each light layer. A rough tomographic reconstruction of the light field on the panel surface is then created by geometrically tracing and triangulating all attenuations detected in the received light. This is said to result in data regarding the position and size of each contact area.
US2009 / 0153519 describes a panel capable of conducting signals. A 'tomograph' is placed next to the panel with signal flow ports lined up around the edge of the panel in discrete positions. Signals (b) measured at the signal flow ports are processed tomographically to generate a two-dimensional representation (x) of the conductivity of the panel, whereby touching objects on the panel surface can be detected. The presented technique for tomographic reconstruction is based on a linear model for the tomographic system, Ax = b. The system matrix A is calculated at the factory, and its pseudoinverse A '<sup>1 </sup>is calculated with truncated SVD algorithms and
535 005 is applied to the measured signals to give a two-dimensional (2D) representation of the conductivity: x = A '<sup>l</sup>b. The proposed process is both very demanding in terms of machining and lacks the suppression of high frequency components, which can lead to a lot of noise in the 2D representation.
US2009 / 0153519 also generally refers to CT (CT). CT methods are well-known imaging methods that have been developed for medical purposes. CT methods use digital geometry processing to reconstruct an image of the inside of an object based on a large series of projection measurements through the object. Various CT methods have been developed to allow efficient processing and / or accurate image reconstruction, such as Filtered Back Projection, ART, SART, etc. Often the projection measurements are performed according to a standard geometry given by the CT method. Obviously, it would be desirable to be able to use existing CT methods to reconstruct the 2D distribution of an energy-related parameter (light, conductivity, etc.) on a touch surface based on a set of projection measurements.
Summary
It is an object of the invention to allow touch determination on a panel based on projection measurements using existing CT methods.
Another object is to provide a technique that allows the determination of touch-related data with sufficient precision to distinguish a plurality of objects which are in simultaneous contact with a touch surface.
This and other objects, which will become apparent from the description below, have been achieved, at least in part, by a method of enabling touch determination, a computer program product, a device allowing touch detection, and a touch-sensitive device according to the independent claims, the independent claims defining embodiments thereof.
A first aspect of the invention is a method for enabling touch determination on the basis of an output signal from a touch-sensitive apparatus, which comprises a panel designed to conduct signals from a plurality of peripheral switching points to a plurality of peripheral switching points and thereby define actual detection lines extending across a surface area of the panel between pairs of switch-on and switch-off points, at least one signal generator connected to the switching points for generating the signals, and at least one signal detector connected to the switching points for generating the output signal. The method comprises: processing the output signal for generating a set of data elements, the data elements indicating detected energy for at least a subset of the actual detection lines; processing the set of data elements for generating one
535 005 set of matched elements, the matched elements indicating estimated detected energy for fictitious detection lines having a surface area position that matches a standard tomographic reconstruction geometry; and processing the set of matched elements by tomographic reconstruction for generation of data indicating a distribution of an energy-related parameter within at least a portion of the surface area.
In one embodiment, the step of processing the output signal comprises: generating the data elements in a two-dimensional measurement space, each data element representing an actual detection line and being defined by a signal value and two dimensional values determining the position of the actual detection line on the surface area.
According to one embodiment, the step of processing the set of data elements comprises: generating estimated signal values for the matched elements at predetermined positions in the two-dimensional measurement space, the predetermined positions corresponding to the fictitious detection lines. The estimated signal values can be generated by interpolation based on the signal values for the data elements, and each estimated signal value can be generated by interpolating the signal values for adjacent data elements in the two-dimensional measurement space.
According to one embodiment, the step of processing the set of data elements further comprises: obtaining a predetermined two-dimensional interpolation function with nodes corresponding to the set of data elements, and calculating the estimated signal values according to the interpolation function and based on the signal values of the data elements. The method may further comprise a step of receiving exclusion data identifying one or more data elements to be excluded, wherein the step of processing the data elements comprises identifying the node corresponding to each data element to be excluded, reshaping the predetermined interpolation function without any such identified and calculating the estimated signal values according to the transformed interpolation function and based on the signal elements of the data elements in the nodes of the transformed interpolation function.
In one embodiment, the matched elements are arranged in rows and / or columns in the two-dimensional measurement space. The matched elements can be arranged at equidistant distances within each of said rows and / or columns.
In one embodiment, the step of processing the set of matched elements comprises: applying a one-dimensional high-pass filtering to the matched elements of the two-dimensional measurement space to generate filtered elements, and processing the filtered elements to generate a set of rear projection values indicating said distribution.
According to one embodiment, the surface area defines a measurement area in the two-dimensional measurement space, and the step of machining comprises the steps, if the actual
535 The 005 detection lines given by the geometrical location of the switch-on and switch-off points result in at least one contiguous region without data elements in the measurement range: obtaining a predetermined set of estimated measurement points within the contiguous region, and, for each estimated measurement point, identifying the position of a corresponding fictional region on the surface area; identifying, for each intersection point, between the corresponding fictive detection line and the actual detection lines and / or between the corresponding fictive detection line and the fictitious detection lines for the set of matched elements, a crossing point value as the smallest signal value of all data elements corresponding to the intersectional points; and calculating a signal value for the estimated measurement point as a function of the intersection point values. In an implementation, the signal value for the estimated measurement point can be given by the largest intersection point value. In another implementation, the method further comprises, for each estimated measurement point: identifying a number of local peaks in the intersection point values, and calculating the signal value of the estimated measurement point as a combination of said local peaks.
In one embodiment, the dimension values comprise a rotation angle of the detection line in the plane of the panel, and a distance to the detection line in the plane of the panel from a predetermined origin.
According to another embodiment, the dimension values include an angle position of the detection line switch-on or disengagement point, and a rotation angle of the detection line in the plane of the panel. In one implementation, the standard geometry is a spring geometry, the touch surface has a non-circular circumference, and the angular position is defined by an intersection between the detection line and a fictitious circle arranged to surround the touch surface.
In one embodiment, the standard geometry is one of a parallel geometry and a spring geometry.
According to one embodiment, the signals comprise one of electrical energy, light, magnetic energy, sound energy and vibrational energy.
In one embodiment, the panel defines a touch surface and an opposing surface, said at least one signal generator being provided to provide light within the panel, in such a way that the light propagates from the switching points by internal reflection between the touch surface and the opposite surface to the switching points to be detected a signal detector, and wherein the touch-sensitive apparatus is designed in such a way that the propagating light is attenuated locally by one or more touching surface objects.
535 005
A second aspect of the invention is a computer program product comprising program code which, when executed in a data processing system, is adapted to perform the method of the first aspect.
A third aspect of the invention is a device which allows touch determination based on an output of a touch-sensitive apparatus, which comprises a panel designed to conduct signals from a plurality of peripheral switching points to a plurality of peripheral switching points and thereby define actual detection lines extending across a surface area of the panel between pairs of switch-on and switch-off points, means for generating the signals at the switch-on points, and means for generating the output signal based on detected signals at the cut-off points. The device comprises: means for receiving the output signal; means for processing the output signal for generating a set of data elements, the data elements indicating detected energy for at least a subset of the actual detection lines; means for processing the set of data elements for generating a set of matched elements, the matched elements indicating an estimated detected energy for fictive detection lines having a position in the surface area matching a standard geometry for tomographic reconstruction; and means for processing the set of matched elements by tomographic reconstruction to generate data indicating a distribution of an energy-related parameter within at least a portion of the surface area.
A fourth aspect of the invention is a touch-sensitive apparatus comprising: a panel designed to conduct signals from a plurality of peripheral switching points to a plurality of peripheral switching points, thereby defining actual detection lines extending across a surface area of the panel between pairs of switching on and off points ; means for generating the signals at the switching points; means for generating an output signal based on detected signals at the cut-off points; and the device which permits touch determination according to the third aspect.
A fifth aspect of the invention is a touch-sensitive apparatus comprising: a panel designed to conduct signals from a plurality of peripheral switching points to a plurality of peripheral switching points, thereby defining actual detection lines extending across a surface area of the panel between pairs of switching on and off points ; at least one signal generator coupled to the switching points for generating the signals; at least one signal detector coupled to the cut-off points to create an output signal; and a signal processor coupled to receive the output signal and designed to: process the output signal for generating a set of data elements, the data elements indicating detected energy for at least a subset of the actual detection lines, processing the set of data elements for generating a set of data elements.
535 005 set of matched elements, the matched elements indicating estimated detected energy for fictitious detection lines having a surface area position that matches a standard tomographic reconstruction geometry, and processing the set of matched elements through tomographic reconstruction for data generation indicating a distribution of an energy-related parameter within at least part of the surface area.
Each of the embodiments of the first aspect may be combined with the second to fifth aspects.
Further objects, features, aspects and advantages of the present invention will become apparent from the following detailed description, the appended claims and the drawings.
Brief description of the drawings
Embodiments of the invention will now be described in greater detail with reference to the accompanying schematic drawings.
Figure 1 is a plan view of a touch sensitive apparatus.
Figures 2A - 2B are top plan views of a touch sensitive apparatus with alternating emitters and sensors respectively.
Figures 3A - 3B are side and top plan views of a touch-sensitive system that uses frustrated total internal reflection (FTIR).
Figure 4A is a flowchart of a method of reconstruction, and Figure 4B is a block diagram of a device implementing the method of Figure 4A.
Figure 5 illustrates the underlying principle of the projection-slice theorem.
Figure 6 illustrates the utility of filtration in rear projection machining.
Figure 7 illustrates a parallel geometry used in tomographic reconstruction.
Figures 8A-8H illustrate a starting point, intermediate result, and end result for a rear projection process when a parallel geometry is used.
Figure 9 illustrates a solar spring geometry used in tomographic reconstruction.
Figures 10A-10C illustrate intermediate and final results for a rear projection process when a solar spring geometry is used.
Figure 11 is a graph with projection values collected in the spring geometry of Figure 9 mapped to a measurement space for a parallel geometry.
Figure 12A is a graph of measurement points defined by the alternate arrangement of Figure 2A; Figures 12B-12C illustrate differences between
535 005 detection lines at an alternate arrangement and a spring geometry, and Figure 12D is a graph of measurement points for the non-alternating arrangement of Figure 2B.
Figure 13 is a reference image mapped to an alternate arrangement.
Figure 14A is a graph with a 2D interpolation function for an alternate arrangement, Figure 14B illustrates the generation of interpolation points using the interpolation function of Figure 14A, Figure 14C is an interpolated sinogram generated from the reference image of Figure 13, and Figure 14D is a reconstructed dämpningsfalt.
Figure 15 illustrates an alternative method for generating interpolation points using the interpolation function of Figure 14A.
Figures 16A-16D and Figures 17A-17B illustrate how the 2D interpolation function is updated when measurement points are removed from the reconstruction.
Figure 18 is a reference image mapped to a non-alternating arrangement.
Figures 19A-19B illustrate a first variant for reconstruction in a non-alternating arrangement.
Figures 20A-20B illustrate a second variant for reconstruction in a non-alternating arrangement.
Figures 21A-21B illustrate a third variant for reconstruction in a non-alternating arrangement.
Figures 22A-22B illustrate a fourth variant for reconstruction in a non-alternating arrangement.
Figures 23A-23F illustrate a fifth variant for reconstruction in a non-alternating arrangement.
Figures 24A-24E illustrate a sixth variant for reconstruction in a non-alternating arrangement.
Figure 25 is a flowchart of a filtered rear projection process.
Figures 26A-26B illustrate a first variant for reconstruction in an alternate arrangement using a tomographic algorithm designed for solar spring geometry.
Figures 27A-27B illustrate a second variant for reconstruction in an alternate arrangement using a tomographic algorithm designed for spring geometry.
Figure 28 illustrates the use of a circle for defining a two-dimensional measurement space for a touch-sensitive device.
Figures 29A-29D illustrate a third variant for reconstruction in an alternate arrangement using a tomographic algorithm designed for solar spring geometry.
535 005
Figure 30 shows the reconstructed attenuation field of Figure 23F after image enhancement processing.
Detailed description of examples of embodiments
The present invention relates to techniques which enable the recovery of touch data for at least one object, and typically a plurality of objects which are in contact with the touch surface of a touch sensitive device. The description begins with a presentation of the underlying concept for such a touch-sensitive device, in particular a device that works with frustrated total internal reflection (FTIR) of light. Following is an example of an overall method for extracting touch data, which includes tomographic reconstruction. The description goes on to generally explain and exemplify the theory behind tomographic reconstruction and its use of standard geometries. Finally, various other innovative aspects of using tomographic reconstruction techniques for touch determination are explained and exemplified.
Throughout the description, the same reference numerals are used to identify corresponding elements.
1st Touch sensitive device
Figure 1 illustrates a touch sensitive apparatus 100 which is based on the concept of transmitting energy of some kind across a touch surface 1, so that an object brought near, or in contact with, the touch surface 1 causes a local decrease in the transmitted energy. The touch-sensitive apparatus 100 includes an arrangement of emitters and sensors which are distributed along the periphery of the touch surface. Each pair of an emitter and sensor defines a detection line which corresponds to the propagation path of an emitted signal from the emitter to the sensor. In Fig. 1, only one such detection line D extending from emitter 2 to sensor 3 is illustrated, although it is understood that the device typically defines a dense network of intersecting detection lines, each detection line corresponding to a signal emitted from an emitter and detected by a sensor. . Thus, any objects touching the touch surface along the extent of the detection line D will reduce its energy when measured by the sensor 3.
The arrangement of sensors is electrically coupled to a signal processor 10, which samples and processes an output signal from the arrangement. The output signal indicates the received energy at each sensor 3. As explained below, the signal processor 10 may be designed to process the output with a tomographic technique to reproduce an image of the distribution of an energy-related parameter (for simplicity called "energy distribution" hereinafter) across the touch surface 1. . The energy distribution can be processed
535 005 further by the signal processor 10 or by a separate device (not shown) for touch determination, and this processing may include the recovery of touch data, such as a position (e.g., x, y coordinates), a shape or surface for each touching object.
In the example of Figure 1, the touch-sensitive apparatus 100 also includes a control unit 12 which is coupled to selectively control the activation of the emitters 2. The signal processor 10 and the control unit 12 may be configured as separate units, or they may be incorporated into a single unit. One or both of the signal processor 10 and the controller 12 can be implemented at least in part by software executed in a processing unit.
The touch-sensitive device 100 may be designed for use with a display device or monitor, for example as described in the Background section. Generally, such a display device has a rectangular width, and thus the touch-sensitive apparatus 100 (the touch surface 1) is also probably designed with a rectangular shape. Furthermore, the emitters 2 and the sensors 3 all have a stationary position around the circumference of the touch surface 1. Thus, unlike a conventional tomographic apparatus used in, for example, the medical field, it will not be possible to rotate the entire measurement system. As described in more detail below, this leads to some limitations on how standard tomographic techniques can be used to recreate / reconstruct the energy distribution within the touch surface 1.
In the following, embodiments of the invention will be described in relation to two main arrangements of emitters 2 and sensors 3. A first major arrangement, shown in Figure 2A, is alternately referred to as having emitters 2 and sensors 3 located one after the other along the periphery of the touch surface 1 . Thus, each emitter 2 is located between two sensors 3. The distance between adjacent emitters 2 is the same along the periphery. The same is true for the distance between adjacent sensors 3. A second main arrangement, shown in Figure 2B, is not alternately named and has only sensors 3 on two adjacent sides (ie, sides meeting in a home), and only emits 2 on its other pages.
The alternate location may be preferred as it creates a more uniform distribution of detection lines. However, there are electro-optic aspects of the alternate system that can speak for the use of the non-alternate placement. For example, the alternate location may require the emitters 2, which can be fed with high driving currents, to be located near the sensors 3, which are designed to detect weak photo currents. This can lead to undesirable detection noise. The electrical connection of the emitters 2 and the sensors 3 can also be somewhat demanding since the emitters 2 and the sensors 3 are spread around the periphery of the touch surface 1. So it can
535 005 is a reason to use a non-alternate placement instead of an alternate placement, since the former eliminates these potential difficulties.
It should be understood that there are many variants and mixtures of these two types of arrangements. For example, the distance / distances sensor-sensor, sensor-emitter, emitter-emitter may vary along the periphery, and / or the mixture of emitters and sensors may be different, for example, there may be two or more emitters / sensors between each emitter / sensor, etc. . Although the following examples are given for the first and second main arrangements, more specifically a rectangular point surface with the 16: 9 image format, this is by way of example only, and the concepts of the invention can be applied regardless of the image format, the point surface shape, and the position of emitters and sensors.
In the embodiments shown herein, at least a subset of the emitters 2 may be arranged to emit energy in the form of a beam or wave diverging in the plane of the touch surface 1, and at least a subset of the sensors 3 may be arranged to receive energy over a large angular span ( point of view). Alternatively or additionally, the individual emitters 2 may be designed to emit a set of separate beams propagating to a number of sensors 3.1. Both embodiments each emitter 2 emit energy to a plurality of sensors 3, and each sensor 3 receives energy from a plurality of emitters 2.
The touch-sensitive apparatus 100 may be designed to allow transmission of energy in one of many different forms. Thus, the transmitted signals can be any radiation or wave energy that can travel in and across the touch surface 1 including, without limitation, light waves in the visible or infrared or ultraviolet range, electrical energy, electromagnetic or magnetic energy, or sound and ultrasonic or vibrational energy.
In the following, an example of an embodiment based on propagation of light will be described. Figure 3A is a side view of a touch-sensitive apparatus 100 which includes a light transmissive panel 4, one or more light emitters 2 (one shown) and one or more light sensors 3 (one shown). Panel 4 defines two opposing and generally parallel surfaces 5,6 and may be flat or curved. A radiation propagation channel is arranged between the two interfaces 5,6 of the panel 4, at least one of the interfaces permitting the propagating light to interact with a touching object 7. Typically, the light from the emitter / emitters 2 propagates through total internal reflection (TIR) in the radiation propagation channel, are arranged at the periphery of the panel 4 to generate a respective measurement signal indicating the energy of the received light.
As shown in Figure 3A, the light can be switched in and out of the panel 4 directly via the portion of the edge portion connecting the upper and lower surfaces 5,6 of the panel 4. Alternatively, not shown, a separate coupling element (e.g. in the form of a wedge) may be attached to the edge portion or to the upper or lower surface 5, 6 of the panel 4 to engage the light into and / or out of the panel 4.
535 005
When the object 7 is brought sufficiently close to the interface, part of the light can be scattered by the object 7, part of the light can be absorbed by the object 7, and part of the light continues to propagate unaffected. Thus, the total internal reflection is frustrated and the energy in the transmitted light is reduced when the object 7 touches an interface on the panel (e.g. the upper surface 5). This type of touch-sensitive device is referred to as FTIR (FTIR Frustrated Total IntemReflection) system in the future.
The touch-sensitive apparatus 100 can be driven to measure the energy of the light transmitted through the panel 4 on a plurality of detection lines. This can be accomplished, for example, by activating a set of spaced emitters 2 to generate a corresponding number of light layers within the panel 4, and by operating a set of sensors 3 for measuring the transmitted energy of each light layer. Such an embodiment is illustrated in Figure 3B, where each emitter 2 creates a light beam that expands in the plane of the panel 4 as it propagates away from the emitter 2. Each beam propagates from one or more entry or switching points within a switching area on the panel 4. Rows of light sensors 3 are positioned around the circumference of the panel 4 to receive light from the emitters 2 at a plurality of separated cut-off points within a cut-off area of the panel 4. It should be appreciated that the switch-on and switch-off points only refer to the position where the beam enters and leaves the panel 4, respectively. Thus, an emitter / sensor may be optically coupled to a number of switch-on / switch-off points. In the example of Figure 3B, however, the detection lines are defined by individual emitter-sensor pairs.
The light sensors 3 together provide an output signal which is received and sampled by the signal processor 10. The output signal contains a number of sub-signals, also called projection signals, each representing the energy of the light emitted by a particular light emitter 2 and received by a particular light sensor 3. , that is, the received energy on a certain detection line. Depending on implementation, the signal processor 10 may need to process the output for identification of the individual sub-signals. Regardless of implementation, the signal processor 10 is capable of obtaining a collection of measurement values containing information about the distribution of an energy-related parameter across the touch surface 1.
The light emitters 2 may be any device capable of emitting light within a desired wavelength range, for example, a diode laser, a vertical-cavity surface-emitting diode (VCSEL), or alternatively an LED (light-emitting diode), a bulb, a halogen lamp, etc.
The light sensors 3 can be any type of device capable of detecting the energy of the light emitted from the set of emitters, such as a
535 005 photodetector, an optical detector, a photoresist, a solar cell, a photodiode, a back-lit LED that acts as a photodiode, a CCD (charge-coupled device), etc.
The emitters 2 can be activated sequentially so that the received energy is measured by the sensors 3 for each light layer separately. Alternatively, all or a subset of the emitters 2 can be activated simultaneously, for example, by modulating the emitters 2 so that the light energy measured by the sensors 3 can be divided into sub-signals by a corresponding de-modulation.
To return to the emitter-sensor arrangement of Figure 2, the distance between adjacent emitters 2 and sensors 3 in the alternate location (Figure 2A) and between adjacent emitters 2 and adjacent sensors 3 in the non-alternate location (Figure 2B) is generally from about 1 mm to about 20 mm. For practical purposes as well as resolution purposes, the distance is generally in the range of 2-10 mm.
In a variant of the alternate placement, the emitters 2 and the sensors 3 may partially or completely overlap, seen in plan view. This can be accomplished by placing emitters 2 and sensors 3 on either side of panel 4, or in some equivalent optical arrangement.
It should be understood that Figure 3 illustrates only one example of an FTIR system. Further examples of FTIR systems are known, for example, from US6972753, US7432893, US2006 / 0114237, US2007 / 0075648, W02009 / 048365, W02010 / 006882,
WO2010 / 006883, WO2010 / 006884, W02010 / 006885, W02010 / 006886, and International Application No. PCT / SE2009 / 051364, all of which are incorporated herein by this reference. The concept of invention is advantageously applicable to such alternative FTIR systems as well.
2nd Transmission
As indicated in Figure 3A, the light is not blocked by the affected object 7. Thus, if two objects 7 happen to be placed one after the other along a light path from an emitter 2 to a sensor 3, part of the light will interact with both objects 7. Provided If the light energy is sufficient, a residual light will reach the sensor 3 and give rise to an output signal which allows identification of both interactions (the touch points). Thus, in the multi-touch type (multi-touch type) FTIR system, the transmitted light may contain information about a plurality of touches.
In the following, the 7) transmission for the jth detection line is T<sub>v</sub> the transmission at a specific position along the detection line and A<sub>v</sub> the relative attenuation at the same point. Thus, the total transmission (modeled) along a detection line is:
535 005
VV
The above equation is suitable for analyzing the attenuation caused by discrete objects on the touch surface when the points are relatively large and separated by a distance. But a more accurate definition of attenuation by an attenuating medium can be used:
O.>
-fa (x) dx = e <sup>J</sup>
In this formulation, / represents the transmitted energy on detection line Dj with attenuating objects, θ 1 represents the transmitted energy on detection line Dj without attenuating objects, and a (x) is the attenuation coefficient along the detection line Dj. We also let the detection line interact with the touch surface along the full extent of the detection line, ie the detection line is represented as a mathematical line.
In order to facilitate the tomographic reconstruction as described below, the measured values can be divided by a respective background value. By appropriate selection of background values, the measured values are thus converted to transmission values, which thus represent the part of the available light energy that has been measured on each of the detection lines.
The theory behind the Radon transform (see below) has to do with line integrals, and it may therefore be appropriate to use the logarithm for the above expression:
log (T) = log (e / "<sup>(X) dx</sup>) = - J a (x) dx
3rd Reconstruction and recovery of contact data
Figure 4A illustrates one embodiment of a method of reconstruction and contact data recovery in an FTIR system. The method includes a sequence of steps 4048 which is performed repeatedly, typically by the signal processor 10 (Figures 1 and 3). Within the scope of this description, each sequence of steps 40-48 is referred to as a sensing event.
Each sensing event starts with a data acquisition step 40, in which measurement values are sampled from the light sensors 2 in the FT1R system, typically by sampling a value from each of the aforementioned sub-signals. The data collection results in a projection value for each detection line. It should be noted that said data can, but does not have to, be collected for all available detection lines in the FTIR system. The data acquisition step 40 may also include preprocessing of the measurement values, e.g., noise reduction filtering,
535 005 conversion of measurement values to transmission values (or, equivalently, attenuation values), conversion to logarithmic values, etc.
In a recalculation step 42, a set of projection values is processed to generate an updated set of projection values that represent fictitious detection lines with a position on the touch surface that matches a standard tomographic reconstruction geometry. This step typically involves interpolation among the projection values as positioned in a 2D measurement space defined by two dimensions, representing the unique location of the detection lines on the touch surface. In this context, a location refers to the physical extent of the detection line on the touch surface as seen in a plan view. The recalculation step 42 is further explained and justified in section 6 below.
In a filtering step 44, the updated set of projection values is subjected to a filtering which aims to increase high spatial frequencies relative to low spatial frequencies among the set of projection values. Thus, step 44 results in a filtered version of the updated set of projection values, hereinafter referred to as the filtered set. Step 44 typically involves applying a suitable 1D filter core to the updated set of projection values. The use of filter cores is further explained and justified in section 4 below.
In a reconstruction step 46, an attenuation field (attenuation field) is reconstructed over the touch surface by machining the filtered set in the 2D measurement space. The damping field is a distribution of damping values over the touch surface (or a relevant part of the touch surface), ie an energy-related parameter. As used herein, the attenuation field and attenuation values can be given in absolute values, such as light energy, or in relative values, such as relative attenuation (e.g., the attenuation coefficient mentioned above) or relative transmission. Step 46 may include applying a rear projection operator to the filtered set of projection values in the 2D measurement space. Such an operator typically generates a single attenuation value by calculating some sort of weighted sum of the selected projection values included in the filtered set. The use of a rear projection operator is explained and further motivated in sections 4 and 5 below.
The damping field can be reconstructed within one or more sub-areas of the touch surface. The subareas can be identified by analyzing intersections between detection lines across the touch surface, based on the above projection signals. Such a technique for identifying subareas is further described in applicant's provisional U.S. Application No. 61 / 272,665, filed October 19, 2009 and incorporated herein by reference.
In a subsequent recovery step 48, the reconstructed attenuation field is processed for identification of touch-related features and recovery of contact data. Each
535 005 prior art can be used to isolate genuine (actual) touch points within the damping field. For example, ordinary blob detection and tracking techniques (tracking) can be used to find the actual touch points. According to one embodiment, a threshold value is first applied to the damping field to remove noise. All regions having attenuation values exceeding the threshold value can be further processed to find the center and shape by adapting, for example, a two-dimensional second-order polynomial or a Gaussian clock shape to the attenuation values, or by locating the attenuation value inertia. There are also a number of other techniques well known in the art, such as clustering algorithms, edge detection algorithms, etc.
Any available touch data can be extracted, including but not limited to the x, y coordinates, areas, shapes and / or pressures of the touch points.
After step 48, the recovered touch data is output, and the processing returns to the data collection step 40.
It will be appreciated that one or more of steps 40-48 may be performed simultaneously. For example, the data acquisition step 40 for a subsequent sensing event can be initiated simultaneously with any of the steps 42-48. It should also be noted that the recalculation and filtering steps 42, 44 can be merged into a single step, since these steps generally involve linear operations.
The process for recovering touch data is typically performed by a data processing device (compare signal processor 10 in Figures 1 and 3) coupled to sample the measurement values from the light sensors 3 in the FTIR system. Figure 4B shows an example of such a data processing device 10 for performing the processing according to Figure 4A. In the example shown, the device 10 includes an input 400 for receiving the output signal. The device 10 further includes a data acquisition element (s) 402 for processing the output signal for the purpose of generating the above set of projection values, and a recalculating element (s) 404 for generating the aforementioned updated set of projection values. A filter element (or member) 406 is also provided for generating the aforementioned filtered set. The device 10 further includes a reconstruction element (s) 408 for generating the reconstructed attenuation field by processing the filtered set, and an output 410 for outputting the reconstructed attenuation field. In the example of Figure 4B, the actual recovery of touch data is performed by a separate device 10 'coupled to receive the attenuation field from the data processing device 10.
The data processing device 10 may be implemented by dedicated software (or firmware) running on one or more general or custom
535 005 special-purpose) calculation devices. In this context, it will be appreciated that each element or organ of such a computational device refers to a conceptual equivalent to a step in the process; there is not always one-to-one correspondence between elements / organs and specific parts of hardware or software routines. A hardware part can sometimes include various means / elements. For example, a processing unit functions as an element / member when executing an instruction, but as another element / member when executing another instruction. In addition, an element / element can be implemented by an instruction in some cases but by a number of instructions in some other cases. Such a software controlled computing device may include one or more processing units, such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application-Specific Integrated Circuit), discrete analog and / or digital components, or any other programmable logic device, such as a Field Programmable Gate Array (FPGA). The data processing device 10 may additionally include a system memory and a system bus which interconnects various system components including the system memory with the processing unit. The system bus can be any of many types of bus structures including memory bus or memory controller, a peripheral bus and a local bus that uses any of a variety of bus architectures. The system memory may include computer storage media in the form of volatile and / or non-volatile memory such as read only memory (ROM), random access memory (RAM) and flash memory. The dedicated software can be stored in the system memory, or on other removable / non-removable volatile / non-volatile computer storage media which are included in or available to the computing device such as magnetic media, optical media, flash memory card, digital tape, semiconductor RAM, semiconductor ROM, etc. . The data processing device 10 may include one or more communication interfaces, such as a serial interface, a parallel interface, a USB interface, a wireless interface, a network adapter, etc., as well as one or more data acquisition devices such as an A / D converter. The dedicated software may be provided to the data processing device 10 on any suitable computer-readable medium, including a disk medium, a read only memory, or an electrical carrier signal.
4th Tomographic techniques
Tomographic reconstruction, which is well known in itself, is based on the mathematics describing the Radon transform and its inverse. The following theoretical discussion is limited to the 2D Radon transform. The general concept of tomography is to image a medium by measuring line integrals through the medium for a large set of angles and positions. The line integrals are measured through the image plane. To find the inverse, that is
535 005 original image, many algorithms use the so-called projection-slice theorem.
Several efficient algorithms have been developed for tomographic reconstruction, such as filtered rear projection, FFT-based algorithms, ART (Algebraic Reconstruction Technique), SART (Simultaneous Algebraic Reconstruction Technique), etc. Filtered rear projection (FBP) is a commonly used algorithm, and there are many variants and developments of it. The following is a brief overview of the underlying mathematics for FBP, solely for the purpose of facilitating the subsequent discussion of the concept of invention and its merits.
4.1 Proiection-slice theorem
Many tomographic reconstruction techniques use a mathematical theorem called the projection-slice theorem. This theorem states that given a two-dimensional function f (x, y), the one- and two-dimensional Fourier transforms and T<sub>2</sub>,<sup>one </sup>projection operator 52 projecting a two-dimensional (2D) function onto a one-dimensional (ID) line, and a slice operator extracting a central slice from a function, the following equations are equivalent:
TJW *, y) = Sr ^ / Uy)
This relationship is illustrated in Figure 5. The right-hand side of the above equation essentially derives a 1D line from the 2D-Fourier transform for function f (x, y). The line passes through the origin of the 2D-Fourier plane, as shown to the right in Figure 5. The left side of the equation starts with projecting (ie integrating along 1 D lines in the projection direction p) the 2D function on a 1 D line (orthogonal to the projection direction p), which creates a projection that consists of the projection values for all the different detection lines. thus extending in the projection direction p. Taking a 1D-Fourier transform of the projection thus gives the same result as taking a slice from the 2D-Fourier transform of the function f (x, y). Within the scope of this description, the function f (x, y) corresponds to the attenuation coefficient field a (x) (generally called attenuation field herein) to be reconstructed.
4.2 Radon Transform
First, it can be noted that the damping disappears outside the touch surface. For the following mathematical discussion, we define a circular disk that encloses the touch surface, Ω<sub>Γ</sub> =
535 005 (x: | x | <r}, with the attenuation field set to zero outside this disk. Furthermore, the projection value for a given detection line is given by:
0 (0, s) = (Jia) (0, s) = fa (x) dx s = x-9
Here we let 0 = (cos φ, sin φ) be a unit vector representing the direction normal to the detection line, and s is the shortest distance (with sign) from the detection line to the origin (taken as the center of the screen, see Figure 5). Notice that 0 is perpendicular to the above projection direction vector, p. This means that we can represent 0 (0, s) with p (<p, s) because the latter designation clearly shows that g is a function of two variables and not a function of a scalar and an arbitrary vector. Thus, the projection value of a detection line could be expressed as p (<p, s), ie as a function of the angle of the detection line toward a reference direction, and the distance from the detection line to an origin. We let the angle span the interval 0 <φ <π, and since the damping field has support in Ω<sub>Γ</sub> then it is sufficient to consider in the interval —r <s <r. The set of projections collected for different angles and distances can be stacked together to form a sinogram.
Our goal now is to reconstruct the damping field a (x) given the measured Radon transform, g = Ka. The radon transform operator is not invertible in the general sense. In order to find a stable inverse, we have to put restrictions on the variations in the damping field.
It should be noted that the Radon transform is the same as the above projection operator in the projection-slice theorem. Thus, taking the 1D Fourier transform of g (<p, s) with respect to the s variable results in central slices from the 2D Fourier transform of the attenuation field a (x).
4.3 Continuous tomography compared to discrete tomography
Previous sections 4.1-4.2 described the mathematics behind tomographic reconstruction using continuous functions and operators. In a real system, however, the measurement data represents a discrete sampling of functions, which requires modification of the algorithms. For a detailed description of such modifications, we refer to the mathematical literature, such as The Mathematics of Computerized Tomography by Natterer, and Principles of Computerized Tomographic Imaging by Kak and Slaney.
535 005
An important modification is the need for a filtering step when operating on discreetly measured functions. The need for flittering can be intuitively understood by considering the projection-slice theorem in a system of discrete measuring points and angles, ie a finite set of detection lines. According to this theorem, for each angle φ, we take the discrete 1D Fourier transform of g (, (p, s) with respect to the variable s and insert the result into the Fourier plane as slices by origin for the 2D-Fourier transform of the original function a ( x) This is illustrated on the left in Figure 6 for a single projection.When we add information from several different projections, the density of measurement points becomes much higher near the origin in the 2D-Fourier transform plane. Since the information density is much higher at low frequencies, an unfiltered rear projection will give a blur from the low frequency components.
To compensate for the unequal distribution of measurement points in the 2D Fourier transform plane, we can increase the amount of information at the high spatial frequencies. This can be achieved by filtration, which can be expressed as the multiplication / weighting of the data points in the 2D Fourier transform plane. This is exemplified on the right in Figure 6, where the amplitude of the high spatial frequencies has been increased and the amplitude of the low frequency components has been decreased. Alternatively, this multiplication in the 2D Fourier transform plane can be expressed as a convolution in the space domain, i.e. with respect to the variable s, using the inverse Fourier transform of the weight function. The multiplication or weight function of the 2DFouriertansform plane is rotationally symmetric. We can thus use the projection lice theorem to obtain the corresponding 1D convolution kernel in the projection domain, ie the kernel that we will use for the projections collected for specific angles. This also means that the folding core will be the same for all projection angles.
4.4 Filtration and rear projection
As explained in the previous section, the sinogram data is filtered first and then rear projection. Filtering can be done by multiplying with a filter Wj, in the Fourier domain. There are also effective ways to implement the filtering as a convolution with a filter w * in the room domain. In one embodiment, the filtering is done only on parameter s and can be described by the following expression:
(Wb * f) (*) = ## (w<sub>island</sub>(s) * g (e, sy) = 32 * v, where 32 * is a rear projection operator defined as:
535 005 π
(J2<sup>#</sup>v) (x) = 2 J ν (θ, χ · θ'ίάφ, ο and W<sub>fc</sub>(x) = 32<sup>#</sup>w<sub>b</sub>. The idea is to choose w<sub>island</sub>(s) filter in such a way that W<sub>hrs</sub>(x) = <5 (x). This is typically accomplished by working in the Fourier domain, taking as a step function with support in a circular disc with radius b, and allowing b - »oo. The corresponding filter in the room domain is = (£) '(<sup>sinc</sup><<sup>ös</sup>) - I (<sup>sinc</sup> (t))) 'with continuous extension along the singularity at s = 0.
In the literature you can find many variants of the filter, such as Ram-Lak, SheppLogan, Cosine, Hann and Hamming.
5th Standard geometries for tomographic processing
Tomographic processing is generally based on standard geometries. This means that the mathematical algorithms require a specific geometric arrangement of the detection lines to achieve the desired precision and / or machining efficiency. The geometric arrangement may be selected to allow determination of the projection values in a 2D measurement space, inter alia, to enable the aforementioned filtration in one of the dimensions of the measurement space prior to the rear projection.
In conventional tomography, the measurement system (i.e., the position of the switching points and / or switching points) is controlled or set to give a desired geometric arrangement of detection lines. Below is a brief presentation of the two main standard geometries used in conventional tomography, for example in the medical field.
5.1 Parallel geometry
The parallel geometry is exemplified in Figure 7. Here, the system measures projection values for a set of detection lines for a given angle <p<sub>fairy</sub>. In Figure 7, the set of detection lines D is indicated by dashed arrows, and the resulting projection is represented by the function 0 (<Pfe, s). The measurement system is then slightly rotated around the origin of the x, y coordinate system in Figure 7, to collect projection values for a new set of detection lines at this new angle of rotation. As shown with the dashed arrows, all detection lines are parallel to each other for each angle of rotation. The system generally measures projection values (line integrals) for angles
535 005 spanning the interval 0 <φ <π. Once all projections have been collected, they can be arranged side by side in a data structure to form a sinogram. The sinogram is generally given in a 2D measurement space defined by dimensions that uniquely assign each projection value a specific line of detection. In the case of parallel geometry, the measurement space is typically defined by the angular parameter φ and the distance parameter s.
Below, the use of a parallel geometry in tomographic processing is further exemplified in relation to a known attenuation field shown in Figure 8A, in which the right bar indicates the coding of grayscale levels for attenuation strength (%). Figure 8B is a graph of the projection values as a function of the distance s of the projection obtained at φ = π / 6 in the attenuation field of Figure 8A. Figure 8C illustrates the sinogram formed by all projections collected from the attenuation field, where the various projections have been arranged as vertical sequences of values. For comparison, the projection shown in Figure 8B is marked as a dashed line in Figure 8C.
The filtration step, i.e., the precipitation, is now done with respect to the variable s, ie in the vertical direction in Figure 8C. As mentioned above, there are many different filter cores that can be used in the filtration. Figure 8D illustrates the central portion of a discrete filter core Wb used in the following examples. As shown, the absolute size of the filter values decreases rapidly from the center of the core (k = 0). In many practical implementations, it is possible to use only the most central parts of the filter core, thereby reducing the number of machining operations in the filtration step.
Since the filtration step is a convolution, it can be computationally more efficient to perform the filtration step in the Fourier domain. For each column with values in the φs plane, a discrete 1D-Fast-Fourier transform is calculated. Then the values thus transformed are multiplied by the 1D-Fourier transform of the filter core. The filtered sinogram is then obtained by calculating the inverse Fourier transform of the result. This technique can reduce the complexity from 0 (n<sup>2</sup>) to 0 (n log<sub>2</sub> (n)) for the filter step for vaije φ, where n is the number of measurement points (projection values) with respect to the variable s.
Figure 8E shows the filtered sinogram obtained by operating the filter core of Figure 8D on the sinogram of Figure 8C.
The next step is to apply the rear projection operator. The basis for the rear projection operator is that a single position in the damping field is represented by a sine function in the sinogram. Thus, to reconstruct each individual attenuation value in the attenuation field, the rear projection operator integrates the values in the filtered sinogram along the corresponding sine function. To illustrate this concept, Figure 8E shows three sine functions P1-P3 corresponding to three different positions in the attenuation field of Figure 8A.
535 005
Since the position of a reconstructed attenuation value will not coincide exactly with all relevant detection lines, it may be necessary to perform a linear interpolation with respect to the variable s where the sine curve intersects between two projection values. Another approach, which is less computationally efficient, is to calculate the filtered values at the intersection points by applying individual filter cores. The interpolation is exemplified in Figure 8F, which is an enlarged view of Figure 8E and in which x indicates the various filtered projection values in the filtered sinogram. The contribution to the rear projection value of the sine curve P1 from the small portion of the φ-s plane shown is:
(1 <sup>- z</sup>2ö) '(<sup>w</sup> * #)26,176 + <sup>z</sup>26 ‘ (<sup>w</sup> * 0)26,177 + (1 ~ <sup>z</sup>27*) (<sup>W</sup> * 0)27,175 + <sup>Z</sup>27 (w * 0) 27.176 + (1 ~ <sup>ζ</sup>2β) '(<sup>w</sup> * 0)28,173 + <sup>Z</sup>28 ' (<sup>w</sup> * 0)28,174
The weights z, in the linear interpolation are given by the normalized distance from the sine curve to the projection value, ie 0 <Zj <1.
Figure 8G shows the reconstructed attenuation field obtained by applying the rear projection operator to the filtered sinogram of Figure 8E. It should be noted that the filtration step is important for the reconstruction to provide useful data. Figure 8H shows the reconstructed attenuation field obtained when the filtration step is omitted.
5.2 Solar spring geometry
Another major type of tomography arrangement is based on measuring data from an individual emitter, instead of measuring parallel projections at several different angles. This so-called fan geometry is exemplified in Figure 9. As can be seen, the emitter emits rays in many directions, and sensors are positioned to measure the received energy of this individual emitter on a number of detection lines D, illustrated by dashed lines in Figure 9. . Thus, the measurement system collects projection values for a set of detection lines D extending from the emitter when placed at an angle βι · In the example shown, each detection line is defined by the angular position β of the emitter with respect to a reference angle (/? = 0 which coincides with x axis), and the angle a of the detection line D with respect to a reference line (in this example a line passing from the emitter through origo). The measurement system is then rotated slightly (δβ) around the origin of the x, y coordinate system in Figure 9, to collect a new set of projection values for this new angular position. It should be noted that the rotation does not need to be limited to 0 <β <π, but can be extended,
535 005 which is well known to those skilled in the art. The following example is given for a complete rotation: 0 <β <
2π.
Solar spring tomographs can be divided into equiangular or equidistant. Equiangular systems collect information at the same angle (seen from the emitter) between nearby sensors. Equiangular systems may be designed with emitters and sensors placed on a circle, or the sensors may be non-equidistant arranged on a line opposite the emitter. Equidistant systems collect information at the same distance between nearby sensors. Equidistant systems can be designed with sensors placed on a line opposite the emitter. The following example is given for an equiangular system and based on the known attenuation field shown in Figure 8A. For a detailed description of the different types of solar spring geometries, we refer to the literature.
Figure 10A illustrates the sinogram formed by all projections collected from the attenuation field of Figure 8A; by the measurement system indicated in Figure 9.1 Figure 10A, the various projections are arranged as vertical sequences of values. It should be noted that the sinogram is given in a 2D measurement space defined by the parameter of the emitter's angular position β and the parameter of the angular direction a.
In an exemplary tomographic processing of the sinogram in Figure 10A, an angular correction is first applied to all projections collected as follows:
β '(«^ βί) = <sup>π</sup> · G (a<sub>hrs</sub>, fii) · cos (a<sub>k</sub>).
The filtration step, i.e., the precipitation, is now done with respect to the variable a<sub>k</sub> in the angularly corrected sinogram, that is, corresponding to the vertical direction of the angularly corrected sinogram. As mentioned above, there are many different filter cores that can be used in the filtration. The following example uses a filter core similar to that shown in Figure 8C. For example, many symmetrical high-pass filters, with a coefficient sum equal to zero, may allow adequate reconstruction of the damping field. However, careful selection of filters may be needed to reduce reconstruction artifacts. The result can also be improved by applying an equalization filter in this step, as is well known in the art. As in the parallel geometry, the filtering may include a convolution in the space domain or a multiplication in the Fourier domain.
The filtered sinogram obtained by operating the filter core on the angularly corrected sinogram is shown in Figure 10B.
The next step is to apply the rear projection operator. The rear projection operator differs from that used in the parallel geometry described above. In the solar spring geometry, the rear projection step can be given by the expression:
535 005 (ft<sup>#</sup>v) (x) = -ip. [| 2 ((<sup>1</sup> “<sup>z</sup>) <sup>v</sup>^<sup>A</sup>k> Pi) + <sup>z</sup> v (a<sub>k +</sub>in, ft)),
Pi where Dj is the position of the source that gives / 5<sub>É</sub>projection, z is a parameter describing the linear interpolation between the detection lines and a beam extending from the source through the position of the attenuation value to be reconstructed.
Figure 10C shows the reconstructed attenuation field obtained by applying the rear projection operator to the filtered sinogram of Figure 10B.
5.3 Sorting algorithms
Another approach for making the filtered rear projection for a spring geometry is to select the locations of emitters and sensors in such a way that it is possible to reorder the data into a parallel geometry. Generally, such sorting algorithms are designed to achieve evenly spaced data elements in the φ-s plane. More information about reordering algorithms can be found in eg Principles of
Computerized Tomographic Imaging by Kak and Slaney.
To further explain the concept of rearrangement, Figure 11 shows the data elements (projection values) collected from two different emitters (ie two different values for β) in an equivangular solar spring tomograph. The data elements are mapped in a φ column. It may be noted that the projection values obtained from an individual emitter do not form a straight vertical line with respect to the variable s. You can also see that the φ values differ by only one constant, and that the s values are identical for the two different projections. Thus, one procedure for reordering is to collect projection values derived from detection lines with the same φ values (ie, from different emitters) and to have these form a column in the φ-s plane. However, this results in non-uniform distances between the s values, which can be overcome by interpolating (re-sampling) the projection values with respect to the variable s. It should be noted that this procedure is a strict LD interpolation and that all columns are subjected to the same transformation. It should also be noted that this procedure transforms one standard tomography geometry into another standard tomography geometry.
For the rearrangement algorithms to work, it is essential (as is said in the literature) that δβ = δα, that is, the angular rotation between two emitter positions is the same as the angular separation between two detection lines. Only when this requirement is met will the projection values form columns with respect to the variable s.
535 005
6th Use of tomographic processing for touch determination
Figure 12A illustrates the measurement points (corresponding detection lines, and thus measured projection values) in the φ-s plane of the alternate system shown in Figure 2A. Due to the irregularity of the measuring points, it is difficult to apply the filter described above. The irregularity of the measuring points also makes it difficult to apply a rearrangement algorithm.
In Figure 12A, the solid lines indicate the physical boundaries of the touch surface. It can be noted that the angle φ actually spans the range from 0 to 2π, since the switching and disengagement points extend around the entire circumference. However, a detection line is the same when rotated with π, and the projection values can thus be rearranged to fall within the range 0 to π. This reorganization is voluntary; the data processing can be done throughout the angular range with a correction of some constants in the rear projection function.
When we compare the alternate arrangement of Figure 2A with the solar spring geometry of Figure 9, we see that the angular positions /?<sub>έ</sub> are not evenly distributed, and that the angular directions a are neither equiangular nor equidistant. In addition, the values obtained for a are different for different βϊ- The different /?; - values for the alternate arrangement are shown in Figure 12B. In an ideal solar spring tomograph, this representation would be a straight line. The step change at emitter 23 is caused by the numbering of the emitter (in this example, the emitter is numbered counterclockwise starting at the lower left corner of Figure 2A). Figure 12C exemplifies the variation of α values for emitter 10 (marked with crosses) and emitter 14 (marked with circles) in figure 2A. In an ideal equilateral solar spring tomograph, this representation would result in two straight lines, with a distance in the vertical direction due to the numbering of the sensors. Instead, Figure 12C shows a lack of regularity both for the individual emitters and between different emitters. Another aspect is that the spring geometry assumes that the source is located, for all projections, at the same distance from the origin, which is not true for an alternate arrangement around a non-circular touch surface.
Figure 12D illustrates the measurement points in the φ-s plane of the non-alternating system shown in Figure 2B. In addition to the irregularity of the measurement points, there are also large parts of the <ps plane that lack measurement points due to the non-alternate placement of switch-on and switch-off points.
Thus, it is not possible to directly apply a filter to the measurement points mapped to a measurement space such as the φ-s plane or the β-α plane, and the measurement points cannot be rearranged to match a standard tomography geometry. This problem is overcome by the recalculation step (42 in Figure 4), which processes the projection values of the measurement points to generate projection values for an updated set
535 005 measuring points. The updated set of measurement points represents a corresponding set of fictitious detection lines. These fictitious detection lines have a location on the touch surface that matches a standard geometry, typically the parallel or solar spring geometry. The generation of projection values for an updated set of measurement points can be achieved by interpolating the original measurement points.
The purpose of the interpolation is to find an interpolation function that can create interpolated values at specific interpolation points in the measurement space based on a set of measured projection values at the original measurement points. The interpolation points, possibly together with some of the original measurement points, form the above-mentioned updated set of measurement points. This updated set of measurement points is generated with a location according to, for example, the parallel geometry or the solar spring geometry. The density of the updated set of measurement points is preferably similar to the average density of the original measurement points in the measurement space.
Many different interpolation functions can be used for this purpose, ie to interpolate data points in a two-dimensional network. The input to such an interpolation function is the original measurement points in the measurement space and the measured projection values for each original measurement point. Most interpolation functions include a linear operation on the measured projection values. The coefficients in the linear operation are given by the known positions of the original measurement points and of the interpolation points in the measurement space. The linear operator can be calculated and then applied to the measured projection values at each sensing time (compare the iterations of steps 40-48 in Figure 4). Some non-limiting examples of suitable interpolation functions include Delaunay triangulation, and other types of interpolations using triangular networks, bicubic interpolation, for example using spline curves or Bezier surfaces, Sinc / Lanczo filtering, nearest neighbor interpolation, and weighted average interpolation.
The following examples are based on Delaunay triangulation, where the measurement points are placed in the hay by a network of non-overlapping triangles. The values of the interpolation points are linearly interpolated in the triangles. The triangles can be calculated using the well-known Delaunay algorithm. In order to obtain triangles of limited skewness, it is often necessary to scale the dimensions of the measurement space (φ, s and β, a) to essentially the same length before applying the Delaunay triangulation algorithm.
The interpolation function will be able to create output values for all given positions in the measurement space. However, the frequency information in the updated set of measurement points will be limited by the density of the original measurement points in the measurement space. Thus, whenever the original density is high, it can
535 005 updated set of measurement points mimic high frequencies present in the measurement data. Whenever the original density is low, as well as if there are large voids in the measurement space, the updated set can only create low frequency variations. Non-alternating arrangements (see Figure 2B) will create a measurement space with one or more contiguous areas (also called empty areas) that lack measurement points (see Figure 12D). These blank areas can be left as they are, or populated with interpolation points, or handled in some other way, as will be explained below in relation to a number of examples.
The following examples will illustrate the recalculation of measurement points to a parallel geometry and a solar spring geometry, respectively. Each example is based on a numerical simulation that starts with a reference image representing a known attenuation field on the touch surface. Based on this known attenuation field, the projection values for all detection lines have been estimated and then used in a tomographic reconstruction according to steps 40-46 in Figure 4, to create a reconstructed attenuation field. Thus, the estimated projection values are used as measured projection values in the following examples.
In the examples, two different merit values are used to compare the quality of the reconstructed damping fields for different embodiments. The first credit value πη is defined as:
- Σ / <sup>mi</sup> ~ 'where f is a reference image (ie, the known attenuation field) and / * is the reconstructed attenuation field. The first credit is intended to capture the similarity between the original image and the reconstructed image.
The second credit value mi is defined as:
<sup>2</sup> ~ s<sub>z</sub>= o | / -r * l 'ie the denominator only includes absolute differences in the regions where the damping value is zero in the reference image. The second advantage is thus intended to capture the noise in the reconstructed image by analyzing the regions in the image where there should be no attenuation.
6.1 Recalculation into a parallel geometry
The following examples will separately illustrate the recalculation into a standard parallel geometry for an alternating arrangement and for a non-alternating arrangement.
535 005 arrangement. Since the recalculation is done for a parallel geometry, the following examples are given for machining in the φ-s plane.
6.1.1 Example: alternate arrangement
This example is given for the alternate arrangement shown in Figure 2A, assuming the reference image shown in Figure 13. The reference image is thus formed by five touching objects 7, of different sizes and attenuation, which are scattered on the touch surface 1. For clarity, Figure 13 shows also the emitters 2 and the sensors 3 in relation to the reference image.
Figure 14A is a plan view of the resulting measurement space, where a network of non-overlapping triangles has been fitted to the measurement points to provide a two-dimensional interpolation function. Figure 14B is a close-up of Figure 14A to illustrate the measurement points (stars) and the Delaunay triangulation (dotted lines extending between the measurement points). Figure 14B also illustrates the interpolation points (circles). The values for the interpolation points are thus calculated by operating the Delaunay triangulation on the projection values in the measurement points. In the example shown, the interpolation points replace the measurement points in the subsequent calculations. In other words, the sinogram formed by the measured projection values is replaced by an interpolated sinogram formed by the interpolated projection values. Thereby, it is possible to obtain a uniform density of interpolation points over the measurement space, if desired. Each interpolation point corresponds to a fictitious detection line extending across the touch surface according to a parallel geometry. The interpolation is thus designed to create a set of fictitious detection lines that match a parallel geometry, allowing a reconstruction of the attenuation field using standard algorithms.
As shown, the interpolation points are arranged as columns (i.e., with respect to the variable s) in the measurement space, which allows subsequent ID filtering with respect to the variable s. and facilitating the subsequent reconstruction processing, for example, the 1D filtration. Preferably, the distance between columns is the same for all columns as this makes the rear projection integral perform better.
In the unpolished sinogram, each φ value together with its associated s values (ie, each column) corresponds to a set of mutually parallel (fictional) detection lines, and thus the data is matched to a parallel geometry in a broad sense.
Figure 14C illustrates the interpolated sinogram, that is, the interpolated projection values calculated by the interpolation function of Figure 14A
535 005 is operated on the measured projection values. After filtering the interpolated sinogram with respect to the variable s, using the filter of Figure 8D, and applying the rear projection operator to the thus-filtered sinogram, a reconstructed attenuation field shown in Figure 14D is obtained, which has the merit values: 1.3577 and m<sub>2</sub>=3.3204.
Figure 15 illustrates an alternative way of creating the updated set of measurement points, again using the Delaunay triangulation (dotted lines). Here, the original measurement points (stars) are retained in the measurement space and supplemented with interpolation points (circles). Around each measurement point, a line of interpolation points with respect to the variable s is generated. This interpolation can of course use the same principles as the previous example. After the interpolation, the 1D filtration is performed locally at each individual measurement value and its complementary interpolation points. In this variant, the updated set of measurement points will exhibit the same variations in density within the measurement space as the set of original measurement points, and it may therefore be advantageous to adjust the bandwidth of the 1D filter to the local density in the measurement space, ie to use different bandwidths for different measurement points. To obtain different bandwidths, we can change the distance, with respect to the variable s, between the supplementary points before calculating the 1D filtering. After filtration, the rear projection operator is applied to the resulting filtered data to calculate a reconstructed attenuation field.
Further variants for generating the updated set of measurement points are of course possible. For example, the above interpolation techniques can be used simultaneously on different parts of the measurement space, or some measurement points can be retained while others are replaced with interpolated points in the updated set of measurement points.
As will be explained below, the generation of the updated set of measurement points may be designed to allow dynamic removal of detection lines during operation of the touch sensitive apparatus. For example, if an emitter or sensor waves poorly, or not at all, during operation of the device, this can have a significant impact on the reconstructed damping field. It is conceivable to provide the apparatus with an ability to identify defective detection lines, for example, by monitoring temporal changes in the light sensors' outputs, and more specifically in the individual projection signals. For example, the temporal changes may appear as changes in the energy / attenuation / transmission or signal-to-noise ratio (SNR) of the projection signals. Defective detection lines can be removed from the reconstruction. Such a touch-sensitive device is described in applicant's provisional U.S. Application No. 61/288416, filed December 21, 2009, and incorporated herein by reference. To take full advantage of such a feature, it can be touch sensitive
535 The 005 apparatus must be designed to have slightly more sensors and / or emitters than are required to achieve sufficient performance, so that it is possible to discard a substantial amount of projection values, for example 5%, without significantly affecting performance.
The recalculation step (compare step 42 in Figure 4) can be designed to dynamically (i.e., for each sensing event) handle such defective detection lines by removing each corresponding measurement point from the measurement space and recalculating the interpolation point whenever a detection line is marked as defective. this measuring point. This will reduce the density of measurement points locally (in the φ-s plane), but the reconstruction processing will continue to function well while discarding information from faulty detection lines.
This is further illustrated in Figures 16-17. Figure 16A is a close-up of a two-dimensional interpolation function formed as an interpolation grid in the measurement space. Assume that this interpolation function is saved for use in the recalculation step for a complete set of measurement points. Also assume that the measurement point indicated by a circle in Figure 16A corresponds to a detection line that is found to be defective. In such a situation, the measurement point is removed, and the interpolation function is updated or recalculated based on the remaining measurement points. The result of this operation is shown in Figure 16B. As shown, the change will occur locally at the triangles closest to the removed measurement point.
If an emitter is judged to be defective, all detection lines starting from that emitter should be removed. This corresponds to the removal of a collection of measurement points and a corresponding update of the interpolation function. Figure 16C illustrates the interpolation function in Figure 16A after such an update, and Figure 16D illustrates the updated interpolation function for the entire measurement space. The removal of the detection lines results in a lower density band (indicated by the arrow L1), but the reconstruction machining still works as it should.
If, instead, a sensor is deemed to be defective, all detection lines starting from this sensor should be removed. This is done in the same way as for the defective emitter, and Figure 17A illustrates the interpolation function of Figure 16A after such an update. Figure 17B illustrates the updated interpolation function for the entire measurement space. Removing the detection lines again results in a lower density band (indicated by arrow L2), but the reconstruction processing works as it should.
6.1.2 Example: non-alternating arrangement
The non-alternate arrangement generally results in a different set of measurement points compared to the alternate arrangement, as seen in Figure 12A with Figure 12D. However, there is no fundamental difference between
535 005 interpolation solutions for these arrangements, and all embodiments and examples of reconstruction machining described above in connection with the alternate arrangement are also applicable to the non-alternate arrangement. The following example therefore focuses on various techniques for handling the empty areas, ie areas without measuring points, which are obtained in non-alternating arrangements.
The following example is given for the non-alternate arrangement in Figure 2B, assuming a reference image shown in Figure 18, ie the same reference image as in Figure 13.
Figure 19A is a plan view of the resulting interpolation function, where a network of non-overlapping triangles has been fitted to the measurement points in the measurement space. This example thus forms the interpolation function directly from the original measurement points. Since the measurement space contains contiguous empty areas (see Figure 12D), the resulting interpolation function is undefined in these empty areas, or alternatively expressed, the values at the implicit measurement points in the empty areas are set to zero. The interpolation function in Figure 19A can be used to generate an updated set of measurement points, as in the previous examples. Figure 19B illustrates the reconstructed attenuation field obtained by calculating the interpolated projection values for the reference image in Figure 18, operating the 1D filter on the result, and applying the back projection operator to the result of the filtered data. The reconstructed damping field has the merit values: mj = 0.7413 and m<sub>2</sub>=1.2145.
An alternative approach to handling the blank areas is to extend the interpolation function over the blank areas, i.e. to extend the grid of triangles over the blank areas, as shown in Figure 20A. Thus, the interpolation function of Figure 19A can be used to generate desired interpolation points throughout the measurement space, ie also in the empty regions. Figure 20B illustrates the interpolated projection values calculated for the reference image in Figure 18. It can be seen that projection values are smeared into the empty areas of the φ-s plane. The reconstructed attenuation field (not shown) obtained after 1D filtration and rear projection has the merit values: mi = 0.8694 and m2 = 1.4532, ie slightly better than Figure 19B.
Another alternative approach is to add some edge nodes to the interpolation function in the empty areas, where these edge nodes form a gradual transition from the original measurement points to zero values, and to let the interpolation function be undefined / zero in the remainder of the empty areas. This results in a smoother transition of the interpolation function into the empty regions, as seen in Figure 21 A. Figure 21B illustrates the interpolated projection values which
535 005 has been calculated for the reference image in Figure 18. The reconstructed attenuation field (not shown) obtained after 1D filtration and rear projection has the merit values: mi = 0.8274 and m2 = 1.4434, ie slightly better than Figure 19B.
All three approaches described above lead to reconstructed damping fields of approximately the same quality. Below is a description of a technique for further improvement of quality, through improved estimation of measurement points in the empty areas.
This improved technique for generating estimated points in the empty areas will be described in relation to Figures 23-24. It should be noted that this technique can also be applied to populate voids formed by the removal of defective detection lines, as a complement or alternative to the technique discussed in Section 6.1.1. In general, the estimated points can be chosen to match the standard geometries, as well as the interpolation points, possibly with a lower density than the interpolation points. Figure 22A illustrates the measurement space supplemented by such estimated points in the empty areas. As in the previous examples, an interpolation function is generated based on the measurement space, in this case based on the combination of measurement points and interpolation points. Figure 22B illustrates the resulting interpolation function.
The purpose is to obtain a good estimate for each added estimate point. This can be achieved by making assumptions about the objects in question, although this is not absolutely necessary. For example, if one can assume that the affected objects are fingertips, one may assume that each of the affected objects results in a cylinder hat profile (top hat) in the damping field with a circular or elliptical contour. Unless the number of touching objects is too large, there will be, for each touching object, at least one detection line that interacts only with that touching object. Assuming that the contact profiles are substantially round, the contact profile will give substantially the same attenuation of all detection lines affected by the contact profile.
The value at each estimation point in the φ-s plane (marked with rhombus in Figure 22A) represents a line integral along a specific line on the touch surface. Since the estimation points are in the empty areas, there is no real (physical) detection line that matches the specific line. Thus, the specific line is a virtual line in the xy plane (ie, a fictitious detection line, although it does not correspond to an interpolation point but an estimate point). The value at the estimation point can be obtained by analyzing selected points along the virtual line in the xy plane. Specifically, a minimum projection value for the selected point is identified by identifying a minimum projection value for the group of detection lines (actual or fictitious) that
535 005 passes through the selected point. This means that for each analyzed point, the algorithm goes through the different detection lines that pass through the point and identifies the lowest value for all these detection lines. The value of the estimation point can then be given by the largest value of all the identified smallest projection values, ie for the various analyzed points, along the virtual line.
To further explain this approach, Figure 23A illustrates the original measurement points together with two estimation points EPI, EP2, which are indicated by circles. The estimation point EP1 corresponds to a virtual line VI, which is indicated in the reference image in Figure 23B. The next step is to evaluate selected points along the virtual line VI. For each selected point, the projection values for all intersecting detection lines are collected. The result is shown in the two-dimensional representation of Figure 23C, which illustrates projection values as a function of detection line (represented by its angle) and the selected points (given as positions along the virtual line). The large black areas in Figure 23C correspond to non-existent detection lines. To find the value of the estimation point EP1, the data in Figure 23C is first processed to identify the smallest projection value (above the angles) of each selected point along the virtual line VI. The result is shown in the graph in Figure 23D. The value of the EP1 estimation point is then chosen as the largest of these smallest projection values. Figure 23E illustrates the values for all estimation points in Figure 22A calculated for the reference image in Figure 18 using this approach, along with the interpolated projection values. By comparing Figure 20B with Figure 21B, one can see a significant improvement with respect to the information in the empty areas of the measurement space. The reconstructed attenuation field, obtained after 1D filtration and rear projection, is shown in Figure 23F and has the merit values: mj = 1.2085 and ni2 = 2.5997, ie much better than Figure 19B.
It is possible to further improve the estimation process. Instead of choosing the largest among the smallest projection values, the processing can identify the presence of multiple touch profiles along the virtual line examined and combine (sum, weighted sum, etc.) the largest projection values for the different touch profiles. To further explain this approach, consider the estimation point EP2 in Figure 23A. The estimation point EP2 corresponds to a virtual line V2, which is indicated in the reference image in Figure 24A. As in the previous example, selected points are evaluated along the virtual line V2. The result is shown in the two-dimensional representation in Figure 24B. As in the previous example, the data in Figure 24B is then processed to identify the minimum projection value (above the angles) of each selected point along the virtual line V2. The result is shown in the graph in Figure 24C. This graph clearly indicates that there are two separate contact profiles
535 005 the virtual line V2. The estimation process thus processes the largest projection values in Figure 24C for the identification of local maxima (in this example two maxima) and sets the value of the estimation point EP2 equal to the sum of said local maxima (projection values). Figure 24D illustrates the values for all estimation points in Figure 22A calculated for the reference image in Figure 18 using this approach, along with the interpolated projection values. The empty areas in the measurement space are represented by relevant information. The reconstructed attenuation field, obtained after 1D filtration and rear projection, is shown in Figure 24E and has the merit values: mi = 1.2469 and m<sub>2</sub>= 2.6589, ie slightly better than Figure 23F.
Figure 25 is a flow chart of an exemplary reconstruction processing, which is a more detailed version of the general processing of Figure 4 adapted for data processing in a touch-sensitive apparatus with a non-alternating arrangement. The processing is done on the output of the light sensor arrangement, using data stored in a system memory 50, and intermediate data generated during the processing. It will be appreciated that said intermediate data may also be temporarily stored in system memory 50 during processing. The flowchart will not be described in more detail as the various steps have already been described above.
In step 500, the processing samples the output of the light sensor arrangement. In step 502, the sampled data is processed to calculate projection values (g). In step 504, the processing reads the interpolation function (IF) from memory 50.
For example, the interpolation function (IF) may be designed as any of the interpolation functions shown in Figures 19A, 20A, 21A and 22B. The processing also reads exclusion data from memory 50, or receives this data directly from a dedicated processing. The exclusion data identifies all defective detection lines to be excluded in the reconstruction processing. The processing modifies the interpolation function (IF) on the basis of the exclusion data, which results in an updated interpolation function (IF).<sup>1</sup>) which can be stored in memory 50 for use during subsequent iterations. On the basis of the updated interpolation function (IF '), and the projection values (g), step 504 generates new projection values (interpolated values, 0 at given interpolation points. Step 504 may also include a calculation of new projection values (estimate values, e) at given estimation points in the empty areas, based on the updated interpolation function (BFj. Step 504 results in a matched sinogram (g, which contains the interpolated values and the estimation values. In step 506, the processing reads the filter core (w<sub>b</sub>) from memory 50 and operating the kernel in a dimension of the matched sinogram (g j. The result of step 506 is a filtered sinogram (υ).
535 In 005 step 508, the processing reads sub-area data from memory 50, or receives this data directly from a dedicated processing. The sub-area data indicates the parts of the damping field / touch surface to be reconstructed. On the basis of the sub-range data, and the filtered sinogram (υ), step 510 creates a reconstructed attenuation field (a), which is output, stored in memory 50 or further processed. After step 508, processing returns to step 500.
It should be appreciated that a similar process can be used for data processing in a touch-sensitive apparatus with an alternating arrangement.
6.2 Recalculation into a solar spring geometry
The following example illustrates the recalculation into a standard solar spring geometry for an alternating arrangement. Since the recalculation is done for a spring geometry, the following examples are given for the / 3-a-plane.
6.2.1 Example: alternate arrangement
This example is given for the alternate arrangement shown in Figure 2A, assuming a reference image shown in Figure 13.
A first implementation of the recalculation step (compare step 42 in Figure 4) will be described with reference to Figure 26.1. The first implementation “compresses” the sampled data to fit a specific solar spring geometry. This means that the projection values obtained for the detection lines in the alternate arrangement are transferred to fictitious detection lines that match a spring geometry, in this example the geometry of an equilateral solar spring tomograph. Making such a transfer may include a step to find the best guess for equilibrium distances at /? The values and for a<sub>k</sub>values. In this example, the measurement points /?<sub>É</sub>values to match the angles of an equiangular solar spring tomograph. This essentially means that the difference in angle of rotation between the different switching points is considered to be the same around the perimeter of the touch surface, ie δβ = 2 · π / Μ, where Af is the total number of emitters (switching points). The measuring points a<sub>k</sub>values are interpreted by postulating that a<sub>fc</sub>values are found at η · δα, where - <n <N and 2N + 1 are the total number of sensors (switch-off points) receiving light energy from the relevant emitter. To get a<sub>k</sub>-values in the correct order, n = 0 can be set as the original sample with the minimum value of a<sub>k</sub>.
Figure 26A illustrates the measurement points in the β-a plane, after this basic transfer of projection values. After angular correction, 1D filtering of angular corrected data and rear projection, a reconstructed attenuation field is shown as shown in Figure 26B. It is obvious that the first implementation is able to recreate
535 005 the original image (Figure 13), but with a rather poor quality, especially in the areas of the hay.
In a second implementation of the recalculation step, the measured projection values are calculated for the calculation of new (updated) projection values for fictive detection lines that match a spring geometry. In the second implementation, as in the first implementation, vatje emitter (connection point) on the perimeter of the touch surface is considered the origin of a set of detection lines in different directions. This means that each /?<sub>f</sub>value corresponds to an emitter (switching point) in the alternating arrangement, which generates a plurality of detection lines with individual angular directions a<sub>k</sub>, and the measurement points defined by the actual & values and a<sub>k</sub> The values thus form columns in the /? - a-plane. Interpolation in the /?, Direction can therefore be omitted and possibly replaced by a step to add an individual weight factor in the rear projection operator (by changing δβ to δβΐ, which should correspond to the difference in vär values between adjacent emitters). In the second implementation, the recalculation step involves an interpolation with respect to a<sub>k</sub>variable, preferably to provide values of interpolation points having equidistant separation with respect to a<sub>k</sub>the variable for each value in the measurement space. Thus, the interpolation of the measured values can be reduced to apply an 1D interpolation function. The interpolation function can be of any type, such as linear, cubic, spline, Lanczos, Sine, etc. In the following example, the interpolation function is linear. However, it should be noted that a 2D interpolation function, as described in section 6.1 above, may alternatively be applied for interpolation in the β-α plane.
Figure 27A illustrates the measurement points in the β-α plane after the 1D interpolation. Figure 27B shows the reconstructed attenuation field obtained after angular correction, 1D filtering of angularly coupled data, and rear projection. By comparing Figure 27B with Figure 26B it can be seen that the second implementation provides a significant quality improvement over the first implementation.
Furthermore, by comparing Figure 27B with Figure 14D, both illustrating reconstructed damping fields for the alternate arrangement, it may appear that the parallel geometry results in a higher reconstruction quality than the spring geometry. This apparent quality difference can have a number of causes. First, the reconstruction algorithm for the solar spring geometry limits the angular direction α to the range - π / 2 <a <π / 2. Angle directions outside this range impair the angle correction (see section 5.2). In the touch sensitive apparatus, detection lines may have angular directions outside this range, especially for emitters located in the corners of the touch surface (where we recall that a = 0 for a line passing from the emitter through origo, ie the center of the touch surface). Secondly, it includes weighted
535 005 rear projection operator (see section 5.2) a normalization based on the inverse of the squared distance between the source and the reconstructed position. This distance becomes close to zero near the perimeter of the touch surface, and its inversion goes towards infinity, thereby reducing the quality of reconstruction at the perimeter. In addition, the standard reconstruction algorithms assume that all sensors (switching points) are located at the same distance from the emitters (switching points).
A third implementation of the recalculating step will now be described with reference to Figures 28-29.1. The third implementation, which is designed to at least partially overcome the above mentioned limitations for the first and second implementations, the detection lines are defined on the basis of fictitious emitter / sensor positions. Figure 28 illustrates the touch-sensitive apparatus enclosed by a circle C, which may but need not be centered on the origin of the apparatus x, y coordinate system (Figure 2). The emitters 2 and sensors 3 provide a set of detection lines (not shown) across the touch surface 1. To define the detection lines in a β-α plane, the intersection between each detection line and the circle C is set to define a by-value, while a<sub>fc</sub>- the value of each detection line is given by the angle of inclination of the detection line relative to a reference line (as in the other examples of solar geometry described herein). Thus, variables β and a are strictly defined in accordance with the theoretical definition illustrated in Figure 9, where variable β is defined as the angle of rotation along a circular circumference.
Figure 29A illustrates the resulting measurement points in the ^ -α plane of the alternate arrangement shown in Figure 28, where /?<sub>É</sub>values are defined according to the previous fictitious-circle approach. The measurement space contains a highly irregular pattern of measuring points. Figure 29B is a plan view of a 2D interpolation function adapted to the measurement points of Figure 29A. It should be understood that the technique described in sections 6.1.1 and 6.1.2 can also be applied to the measurement points in the /? - a plane for generating interpolation / estimation points representing fictive detection lines that match a standard solar spring geometry. Thus, the interpolation / estimation points are conveniently generated to form columns with respect to the variable β, preferably with equidistant distances. Figure 29C illustrates the interpolated sinogram obtained by operating the interpolation function of Figure 29B on the projection values given by the reference image in Figure 13. Figure 29D shows the reconstructed attenuation field obtained after an angle correction, 1D filtering of angle corrected data, and rear projection data. By comparing Figure 29D with Figure 27B, it is seen that the third implementation provides a significant quality improvement over the first and second implementations.
535 005
In all of the above implementations, the recalculation step results in an updated sinogram, in which each β value and its associated α values (i.e., each column of the sinogram) corresponds to a solar spring of detection lines of a common origin, and thus the data is adapted to a solar spring geometry in a in a sense.
7th Concluding remarks
The invention has been mainly described above with reference to a few embodiments. However, those skilled in the art will recognize that embodiments other than those described above are equally possible within the scope and spirit of the invention, which are defined and limited solely by the appended claims.
For example, the reconstructed attenuation field may undergo post-processing before the extraction of the contact data (step 48 in Figure 4). Such post-processing may include various types of filtration, for noise removal and / or image enhancement. Figure 30 illustrates the result after applying a Bayesian image enhancer to the reconstructed attenuation field of Figure 24E. The improved damping field has the merit values: mi = 1.6433 and m2 = 5.5233. In comparison, the improved attenuation field obtained by applying the Bayesian image enhancer to the reconstructed attenuation field in Figure 14D has the merit values: mi = 1.8536 and m2 = 10.0283.1 in both cases, a significant quality improvement is obtained.
Furthermore, it should be appreciated that the concept of the invention is applicable to all touch-sensitive devices which define a fixed set of detection lines and which operate via processing of measured projection values for the detection lines according to any tomographic reconstruction algorithm defined for a standard geometry, where this standard geometry does not match the fixed line array. Thus, while the above description has been given with reference to FBP algorithms, the concept of the invention has a more general use.
It should also be clarified that all of the above embodiments, examples, variants and alternatives with respect to interpolation, deletion of detection lines, and estimation in empty areas are generally applicable to all types of emitter-sensor arrangements and regardless of standard geometry.
Further, the reconstructed attenuation field need not represent the distribution of attenuation coefficient values within the touch surface, but may instead represent the distribution of energy, relative transmission, or any other relevant magnitude that can be obtained by processing the projection values given by the sensor output. Thus, the projection values may represent measured energy, difference energy (e.g., given a measured energy value minus a background energy value for each detection line), relative attenuation, relative transmission, a logarithmic
535 005 attenuation, a logarithmic attenuation, etc. Those skilled in the art will recognize that there are other ways of generating projection values based on the output signal. For example, any individual projection signal included in the output signal may be subjected to a high-pass filtering in the time domain, the filtered projection signals thus representing background compensated energy and can be sampled to generate projection values.
Furthermore, all of the above embodiments, examples, variants and alternatives with respect to an FTIR system are equally applicable to a touch-sensitive apparatus which acts by transmitting energy other than light. In one example, the touch surface may be implemented as an electrically conductive panel, the emitters and sensors being electrodes that switch electrical current into and out of the panel, and the output signal indicating the panel's resistance / impedance on the individual detection lines. In another example, the touch surface may comprise a material acting as a dielectric material, the emitters and sensors being electrodes, and the output signal indicating the panel's capacitance on the individual detection lines. In yet another example, the touch surface may include a material that acts as a vibration-conducting medium, the emitters being vibration generators (e.g., acoustic or piezoelectric transducers), and the sensors being vibration sensors (e.g., acoustic or piezoelectric sensors).
In addition, the concept of the invention can be applied to improve tomographic reconstruction in all fields of technology, such as radiology, archeology, biology, geophysics, oceanography, materials science, astrophysics, etc., whenever the detection lines do not match a standard geometry that forms the basis of the tomographic reconstruction algorithm. Thus, the concept of the invention can generally be defined as a method of image reconstruction based on an output from a tomographer, the tomographer comprising a plurality of peripheral input points and a plurality of peripheral output points defining between themselves actual detection lines extending over a measurement space to propagate energy input signals. to the output points, at least one signal generator coupled to the input points for generating the energy signals; and at least one signal detector coupled to the output signal output points, the method comprising: processing the output signal to create a set of measurement data, wherein the measurement data indicates detected energy for at least a subset of the actual detection lines; processing the set of measurement data for generating a set of matched elements, wherein the matched elements indicate estimated detected energy for fictive detection lines having a position in the measurement space matching a standard geometry for tomographic reconstruction; and processing the set of matched elements by tomographic reconstruction to generate data indicating a distribution of an energy-related parameter within at least a portion of the measurement space.
535 005
Contents3
26 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26
28 members in 11 offices
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 1050434 | Sweden | A | |
| SE20100050434 | – | – | – |
Members28
| Document | Office | Kind | |
|---|---|---|---|
| SE1050434A1 | Sweden | A1 | |
| CA2798176A1 | Canada | A1 | |
| WO2011139213A1 | World Intellectual Property Organization (WIPO) | A1 | |
| TW201203052A | Taiwan Province of China | A | |
| SE535005C2This record | Sweden | C2 | |
| IL222797D0 | Israel | D0 | |
| US2013044073A1 | United States of America | A1 | |
| EP2567306A1 | European Patent Office (EPO) | A1 | |
| CN103026325A | China | A | |
| JP2013527528A | Japan | A | |
| KR20130108077A | Republic of Korea | A | |
| EP2567306A4 | European Patent Office (EPO) | A4 | |
| RU2012148777A | Russian Federation | A | |
| US8780066B2 | United States of America | B2 | |
| US2014267124A1 | United States of America | A1 | |
| JP5807057B2 | Japan | B2 | |
| EP2567306B1 | European Patent Office (EPO) | B1 | |
| EP3012721A1 | European Patent Office (EPO) | A1 | |
| CN103026325B | China | B | |
| CN105930002A | China | A | |
| US9547393B2 | United States of America | B2 | |
| US2017102827A1 | United States of America | A1 | |
| KR101760539B1 | Republic of Korea | B1 | |
| KR20170086137A | Republic of Korea | A | |
| KR101840991B1 | Republic of Korea | B1 | |
| US9996196B2 | United States of America | B2 | |
| US2018253187A1 | United States of America | A1 | |
| CN105930002B | China | B |
Numbers
- Publication, DOCDB
- 535005
- Publication, EPODOC
- SE535005
- Application
- 1050434
- Application, DOCDB
- 1050434
- Application, EPODOC
- SE20100050434
Titles2
- English
- Determination of contact through tomographic reconstruction
- Swedish
- Beröringsbestämning genom tomografisk rekonstruktion
Classification
- CPC, 1
- G06F3/041
- IPC, 2
- G06T11 00
- G06F3 041