Method for locating an impact on a surface and device therefor

0001The present invention relates to methods for locating an impact on a surface and to devices for implementing these methods.

0002More particularly, the invention relates to a method in which an impact is localized on a surface belonging to an object forming an acoustic interface, provided with at least one acoustic sensor (the object forming an acoustic interface can be made in one piece or of several elements, assembled or at least in mutual contact), process in which at least one signal is picked up from acoustic waves generated in the object forming an acoustic interface by said impact and the impact is localized by processing said picked up signal.

0003The document <patcit id="pcit0001" dnum="FR2811107A"><text>FR-A-2 811 107</text></patcit> describes an example of such a method which applies in particular to a window pane. In this known method, the position of the impact on the surface of the object is calculated by measuring the differences in time of flight of the acoustic waves to different sensors.

0004This known process however requires:<ul id="ul0001" list-style="dash" compact="compact"><li>that the glass used has perfect homogeneity and a perfect surface condition,</li><li>that the fields of the glass are specially treated in particular to avoid reflections from the acoustic waves,</li><li>that we know in advance the speed of the acoustic waves in the glass, which implies knowing precisely its composition,</li><li>that at least four sensors are used.</li></ul>

0005As a result, this known method is particularly expensive to implement and cannot be applied to any pre-existing objects, in particular heterogeneous objects made up of assemblies of parts, objects of irregular shape, etc.

0006The present invention aims in particular to overcome these drawbacks.

0007To this end, according to the invention, a method of the kind in question is characterized in that it comprises a recognition step during which the signal picked up is compared with at least one predetermined signal corresponding to the signal which is picked up when an impact is generated on at least one active area belonging to the surface of the object forming an acoustic interface (this comparison, which can be made both in the time domain and in the frequency domain, may possibly be performed on only part of the signal received or on data extracted from the signal received after processing, in which case said predetermined signal can be reduced to the part on which the comparison is made or to the data on which the comparison is made) , and the impact is associated with said active area if the signal received is sufficiently close to said predetermined signal.

0008Thanks to these provisions, an impact positioning process is obtained which is robust, adaptable to all objects (including heterogeneous objects formed by assembling several parts or by bringing several parts into contact), easy and inexpensive to enforce.

0009In preferred embodiments of the invention, it is possible optionally to have recourse to one and / or the other of the following arrangements:<ul id="ul0002" list-style="dash" compact="compact"><li>the surface of the object forming the acoustic interface comprises several active zones, and during the recognition step, the signal picked up is compared with several predetermined signals each corresponding to the signal picked up when an impact is generated on one of said active zones;</li><li>a single acoustic sensor is used;</li><li>several acoustic sensors are used and during the recognition step, a signal is received for each acoustic sensor and the signals picked up by the different acoustic sensors are compared to the predetermined signals independently of each other;</li><li>the signals picked up by the various acoustic sensors are compared to the predetermined signals differently from each other;</li><li>several acoustic sensors are used which measure several different quantities;</li><li>at most two acoustic sensors are used;</li><li>the method comprises an initial learning step during which each predetermined signal is experimentally determined by generating at least one impact on each active area;</li><li>each predetermined signal is a theoretical signal (calculated or determined experimentally on an identical or very similar object from the acoustic point of view to that used);</li><li>during the recognition step, the signal received is compared with said at least one predetermined signal by cross-correlation;</li><li>during the recognition step, the signal received is compared with said at least one signal predetermined by a recognition method chosen from voice recognition, signal recognition, shape recognition, and recognition by neural network;</li><li>during the recognition step, the signal received is associated either with a single active zone, or with no active zone;</li><li>each active zone is associated with predetermined information (for example, an alphanumeric character, a command, etc.) and when the impact is associated with an active zone, the predetermined information corresponding to this active zone is used by a electronic device;</li><li>the surface of the object forming the acoustic interface comprises a number n of active zones, n being at least equal to 2, and the recognition step comprises the following sub-steps:<ul id="ul0003" list-style="bullet" compact="compact"><li>an intercorrelation of the signal received (generally after normalization) is carried out with said predetermined signals Ri (t), i being a natural integer between 1 and n which designates an active area, and thus intercorrelation functions Ci (t ),</li><li>a potentially activated active area j is determined which corresponds to the intercorrelation result Cj (t) having a maximum amplitude higher than those of the other results Ci (t),</li><li>the distribution D (i) of the amplitude maxima of the intercorrelation results is also determined: <maths id="math0001"><math display="block"><mi mathvariant="normal">D</mi><mo>(</mo><mi mathvariant="normal">i</mi><mo>)</mo><mo>=</mo><mi>Max</mi><mrow><mo>(</mo><mfenced><mi>This</mi><mrow><mo>(</mo><mi mathvariant="normal">t</mi></mrow></mfenced><mo>)</mo><mo>,</mo></mrow></math><img file="EP1512116B9_D0001.tif" /></maths></li><li>the distribution D '(i) of the amplitude maxima of the intercorrelation results C'i (t) between Rj (t) and the various predetermined signals Ri (t) is also determined: D' (i) = Max (( C'i (t)),</li><li>it is determined whether the impact has been generated on the active area j as a function of a level of correlation between the distributions D (i) and D '(i);</li></ul></li><li>during the recognition step, the captured signal is processed to extract data representative of certain characteristics of the captured signal and the data thus extracted is compared to reference data extracted from the signal which is picked up when an impact is generated on each active zone;</li><li>during the recognition step, a code is determined from said data extracted from the sensed signal and this code is compared to a table which gives a correspondence between at least certain codes and each active area;</li><li>the object forming the acoustic interface comprises at least two active zones and during the recognition step, resemblance values ​​representative of the resemblance between the sensed signal and the predetermined signals are determined (in particular a value derived from the function of intercorrelation, for example its maximum), the impact is associated with several adjacent active zones corresponding to maximum resemblance, called active reference zones, then the position of the impact on the surface is determined as a function of the similarity values ​​assigned to the active reference zones;</li><li>the position of the impact on the surface is determined so that the similarity values ​​assigned to the active reference zones correspond as closely as possible to theoretical similarity values ​​calculated for said active reference zones for an impact generated in said position on the surface ;</li><li>the theoretical similarity values ​​are functions of the position of the impact on the surface, determined in advance for each possible set of active reference zones;</li><li>the active area is identified by comparison between the phase of the predetermined signals Ri (t) and the signal received;</li><li>during the learning phase, we calculate the Fourier transform R_i(ω) = | Ri (ω) | .e^{j ϕi (ω)} of each acoustic signal R_i(t) generated by an impact on the active area i, where i is an index between 1 and n, and this Fourier transform is only kept with the phase component e^{j ϕi (ω)}, in the only frequency bands ω where the amplitude | Ri (ω) | is greater than a predetermined threshold, then the same processing is applied to each acoustic signal picked up S (t) during normal operation of the device;</li><li>the predetermined threshold is equal to the maximum of MAX / D and | B (ω) |, where:<ul id="ul0004" list-style="bullet" compact="compact"><li>MAX is chosen from the maximum value of the modules | R_i(ω) |, the maximum value of the modules | R_i(ω) | each standardized in energy, and the maximum value of the envelope of the mean of the modules | R_i(ω) | each standardized in energy,</li><li>D is a constant,</li><li>| B (ω) | is the average of several noise spectra in the object forming an acoustic interface, acquired at different times;</li></ul></li><li>during normal operation of the device:<ul id="ul0005" list-style="bullet" compact="compact"><li>we calculate a product P_i(ω) equal to S '(ω) multiplied by the conjugate of R_i'(ω) for references i = 1 ... n,</li><li>then we normalize the P products_i(ω),</li><li>then carry out the inverse Fourier transform of all the products P_i(ω), and we obtain temporal functions X_i(t),</li><li>and the signal S (t) is assigned to an active area (10) as a function of said time functions X_i(t);</li></ul></li><li>the signal S (t) is assigned to an active area as a function of the maximum values ​​of said time functions X_i(t).</li></ul>

0010Furthermore, the subject of the invention is also a device specially adapted for implementing an interfacing method as defined above.

0011Other characteristics and advantages of the invention will appear during the following description of five of its embodiments, given by way of nonlimiting examples, with reference to the accompanying drawings.

0012In the drawings:<ul id="ul0006" list-style="dash" compact="compact"><li>the <figref idref="f0001">figure 1</figref> is a schematic perspective view showing an example of a device comprising an acoustic interface suitable for implementing a method according to a first embodiment of the invention,</li><li>the <figref idref="f0001">figure 2</figref> is a block diagram of the device of the <figref idref="f0001">figure 1</figref>,</li><li>the <figref idref="f0001">figure 3</figref> represents a graph illustrating an example of a method which makes it possible to associate an impact on the surface of the visible acoustic interface on the <figref idref="f0001">figure 1</figref>, with an active area of ​​this surface,</li><li>the <figref idref="f0002">figure 4</figref> schematically represents an acoustic interface usable in a device for implementing a method according to a second embodiment of the invention,</li><li>the <figref idref="f0002">figure 5</figref> is a block diagram of an example of a device that can use the input interface of the <figref idref="f0002">figure 4</figref>,</li><li>and the <figref idref="f0003">figures 6 to 9</figref> schematically represent acoustic interfaces usable in a device for implementing a method according to third, fourth and fifth embodiments of the invention.</li></ul>

0013In the different figures, the same references designate identical or similar elements.

0014The <figref idref="f0001">figure 1</figref> represents a device 1 intended to implement the present invention, which comprises for example:<ul id="ul0007" list-style="dash" compact="compact"><li>a microcomputer central unit 2,</li><li>a screen 3 connected to the central unit 2,</li><li>and an acoustic input interface 4 which makes it possible to communicate information to the central unit 2 in the example considered.</li></ul>

0015The acoustic input interface 4 comprises a solid object 5, constituted here by a table in which acoustic waves are propagated by generating impacts on its surface 9, as will be explained below.

0016It should be noted, however, that the object forming an acoustic interface could consist of any other object, homogeneous or heterogeneous, consisting of a single piece or of several pieces assembled or simply in mutual contact, such as: window, door, window, portable tablet, computer screen, display panel, interactive terminal, toy, vehicle dashboard, rear seat backrest of motor vehicle or airplane seat, wall, floor, vehicle bumper (the information transmitted by the acoustic interface then being the position of an impact on the bumper), etc.

0017At least one acoustic sensor 6 (a single sensor 6 in the example shown) is fixed to the object 5, this acoustic sensor 6 being connected for example to the microphone input 7 of the central unit 2, via a cable 8 or by any other means of transmission (radio, infrared or other), so as to capture said acoustic waves and transmit them to the central unit 2.

0018The acoustic sensor 6 can for example be a piezoelectric sensor, or the like (for example, a capacitive sensor, a magnetostrictive sensor, an electromagnetic sensor, an acoustic velocimeter, an optical sensor [laser interferometer, laser vibrometer, ...] , etc.). It can be adapted to measure for example the amplitudes of the displacements due to the propagation of sound waves in the object 5 forming an acoustic interface, or the speed or the acceleration of such displacements, or even it can be a pressure sensor measuring the pressure variations due to the propagation of the acoustic waves in the object 5.

0019On the external surface 9 of the object 5 (in this case on the upper face of the table constituting said object 5 in the example shown on the <figref idref="f0001">figure 1</figref>), several active zones 10 are defined, which can be delimited for example:<ul id="ul0008" list-style="dash" compact="compact"><li>by a physical marking, removable or not, affixed to the surface 9 of the object 5,</li><li>or by a light marking obtained by projecting an image onto the surface 9.</li></ul>

0020Surface 9 could also have parts where it would be prohibited to generate an impact for example by covering them with a flexible material or simply inaccessible to the user, in particular for better reliability of the system.

0021The different active areas 10 may simply be portions of the surface 9, identical to the rest of the surface 9. These active areas, however, differ from one another and from the rest of the surface 9, insofar as an impact on one of the zones 10 generates an acoustic signal different from the signal generated by an impact on another of the active zones 10 or on another part of the surface 9.

0022Each of the active zones 10 is associated with predetermined information that a user may want to communicate to the central unit 2. The information in question can for example be a command, a number, a letter, a position on the surface 9, or any other information which can usually be transmitted to an electronic device such as a microcomputer (or individually control unit of another electronic device) by means of conventional input interfaces such as keyboards, control buttons, mouse or others.

0023The information in question can possibly be indicated in clear by markings 10a on the surface 9 (as for the markers of the zones 10, these markings can be affixed physically on the surface 9 in a final or removable manner, or they can be projected under form of bright images on said surface 9).

0024As a variant, the surface 9 of the object 5 may simply include markers (physically or lightly affixed) making it possible to distinguish the active zones from one another. These references may for example be numbers or colors, and their meaning may possibly be recalled by a display generated by the central unit 2 on the screen 3.

0025Optionally, the surface 9 may also not include any marking, either to delimit the active areas, or to identify the information to which they correspond, in which case the active areas 10 would only be known to authorized users of the device 1 only.

0026It will be noted that the predetermined information associated with each active zone 10 can either be always the same, or vary depending on the progress of a program in the central unit 2, or even depend on previous actuations of other active zones 10 (some active zones 10 can for example be actuated to change the function assigned to one or more active zone (s) 10 actuated after it, so as, for example, to access specific functions, to special characters, or to put capital letters, etc.).

0027The different active areas 10 of the object 5 therefore constitute a real virtual keyboard which is operated by tapping on the active areas, indifferently with the fingernail of a finger, with the end of the fingers, with an object such as pen, stylus or whatever.

0028It will be noted that the surface 9 of the object 5 could if necessary comprise a single active area 10 in the simplest cases, this active area 10 does not however extend to the whole of the surface 9 and preferably constitutes a small portion of said surface 9.

0029As shown in the <figref idref="f0001">figure 2</figref>, the sensor 6 (SENS.) can conventionally be connected via the input 7 to an amplifier 11 itself connected to an analog-digital converter 12 (A / D) which transmits the signals received to the processor 13 of the central processing unit 2 (CPU) which processor is itself connected to one or more memories 14 (MEM.) and controls the above-mentioned screen 3 (SCR.) or any other output interface returning information to the user.

0030It will be noted that the acoustic interface 4 could serve as an information input interface to all other electronic devices than a microcomputer, for example a household or professional electronic device, a digital code, a central electronic unit of a vehicle, etc. In all cases, the electrical signals generated by the sensor (s) 6 can be processed either in this electronic device, or in an external digital signal processing device (DSP).

0031During the use of the device 1 described above, when a user generates an impact on the surface 9 of the object 5, this impact generates an acoustic wave which propagates in the object 5 to the acoustic sensor 6. The sensor acoustic 6 then generates an electrical signal S (t) which, after digitization, is processed by processor 13 (or by another dedicated processor, internal or external to the central unit 2).

0032The processor 13 then compares the received signal with different predetermined signals belonging to a library of signals previously stored in the memory 14, these predetermined signals corresponding respectively to impacts generated on the different active areas 10 of the object 5.

0033This comparison makes it possible to know whether the acoustic signal comes from one of the active zones 10, and which, whatever the mode of excitation of said active surface (impact of a nail, of a finger end, of a palm, an object such as a pen or stylus, etc.).

0034The predetermined signals from the signal library may have been determined during an initial learning phase in which impacts are successively generated on all the active zones 10 of the object 5, by recording the corresponding signals (preferably after normalization, for example so that the energy of each reference signal is equal to 1) received in the central unit 2 via the acoustic sensor 6.

0035Alternatively, when the object 5 has a simple and / or repetitive geometric shape, it is possible that the predetermined signals from the signal library are obtained by modeling or are determined experimentally only once for all the objects 5 in a series identical objects: in these two cases, there would therefore be no prior learning phase for the particular object 5 connected to the central unit 2, but simply installation of the signal library in the memory 14 of said central unit.

0036It will be noted that in certain cases (in particular if the object 5 is made of wood), it is possible to vary the predetermined signals from the library of signals as a function of the ambient conditions, in particular the temperature and the humidity. These variations can be calculated or result from a new learning phase.

0037The comparison of the signals received during the use of the device 1, with the predetermined signals from the signal library, can be carried out:<ul id="ul0009" list-style="dash" compact="compact"><li>directly on the time signals S (t) received from the sensor 6,</li><li>or also on the frequency spectrum of these signals (for example after Fourier transform of the time signals received from the sensor 6),</li><li>or on other characteristic data of the signal, in particular its phase.</li></ul>

0038The comparison of the signals picked up with the predetermined signals from the signal library can be carried out by any known means, in particular:<ul id="ul0010" list-style="dash" compact="compact"><li>by intercorrelation,</li><li>by known methods of voice recognition, signal recognition or shape recognition,</li><li>by using neural networks, or the like.</li></ul>

0039By way of a more precise example, it is possible in particular to use, for recognizing the active area 10 from which the captured signal S (t) comes, the following method:<ol id="ol0001" compact="compact"><li>(1) After normalization of the signal received S (t) (for example, we calibrate S (t) so that its energy is equal to 1), we proceed to an intercorrelation of the signal S (t) generated by the sensor 6 with the n predetermined signals from the library also normalized, denoted Ri (t) with i = 1..n. We thus obtain functions Ci (t), which are the temporal results of the product of intercorrelation of the signal S (t) respectively with the signals Ri (t) of the library. From these calculations, a potentially activated active area j is determined. J corresponds to the intercorrelation result Cj (t) having a maximum amplitude higher than those of the other results Ci (t).</li><li>(2) The distribution D (i) of the amplitude maxima of the intercorrelation results is also determined: <maths id="math0002"><math display="block"><mi mathvariant="normal">D</mi><mo>(</mo><mi mathvariant="normal">i</mi><mo>)</mo><mo>=</mo><mi>Max</mi><mrow><mo>(</mo><mfenced><mi>This</mi><mo>(</mo><mi mathvariant="normal">t</mi></mfenced><mo>)</mo><mspace width="1em" /><mi>with i</mi><mo>=</mo><mn>1..</mn><mo>⁢</mo><mi mathvariant="normal">not</mi><mn>.</mn></mrow></math><img file="EP1512116B9_D0002.tif" /></maths></li><li>(3) We calculate a second distribution function D '(i) obtained in an identical way to the calculation of the function D (i) but by replacing S (t) by Rj (t).</li><li>(4) We carry out an intercorrelation of the distributions of the maxima of amplitudes D (i) and D '(i). If the maximum amplitude E of the intercorrelation result between D (i) and D '(i) is sufficient, then j is the number considered of the activated zone. Otherwise, the signal generated by the sensor corresponds to a false alarm.</li></ol>

0040During this step (4), we can simply calculate E and the maximum value of D (i), i.e. Max (D (i)): if we consider these two values ​​as the coordinates of a point in a two-dimensional space of axes x = Max (D (i)) and y = E, as shown in the <figref idref="f0001">figure 3</figref>, we can determine in advance (empirically or by calculation) a threshold curve L which delimits a domain D corresponding to the validated points (this domain is finite and limited to x = 1 and y = 1, absolute maximum values ​​of D ( i) and E. The signals received which give points outside the domain D, on the other hand, are eliminated as being false alerts.

0041In the example considered, the line D is a straight line which can pass for example through the points (S1, 0) and (0, S2). For example, S1 = 0.4 and S2 = 0.4 or 0.6.

0042It will be noted that in addition to identifying the active area 10 from which the impact comes, it would be possible to measure the force of the impact, for example to guide the user in his way of using the interface. acoustics, or to modulate the action triggered by an impact on an active area 10, according to the intensity of this impact.

0043It will also be noted that the recognition of the signals coming from the active zones 10 can possibly be done by using only part of the signals S (t) received or part of their frequency spectrum or more generally part of their characteristics. In this case, during the recognition step, the captured signal is processed to extract data representative of certain characteristics of the captured signal and the data thus extracted is compared with reference data extracted from the signal which is picked up when an impact is generated on each active zone.

0044Thus, it is for example possible to measure the amplitude and the phase of the signal for m predetermined frequencies (m being a natural integer at least equal to 1), and to compare these amplitudes measured a1-am and these phases measured p1-pn with the amplitudes Ai1-Aim and the phases Pi1-Pim measured at said predetermined frequencies from the signals received during the learning phase (or determined by modeling ) for the different active zones 10 of number i (i being between 1 and n, where n is the number of active zones 10).

0045As a variant, it is possible to determine a code from said data extracted from the signal received and to compare this code to a table which gives a correspondence between at least certain codes and each active zone (the codes contained in this table then represent the signals from the signal library mentioned above).

0046By way of nonlimiting example, a 16-bit code can be determined from the sensed signal S (t), as follows:<ul id="ul0011" list-style="dash" compact="compact"><li>the first 8 bits of the code are determined from the frequency spectrum of the signal S (t) which is subdivided into 8 predetermined frequency slots [f_k, f_{k + 1}], k = 1..8: the bit of rank k is equal to 1 for example if the final energy value given by the spectrum at frequency f_{k + 1} is greater than the average energy value of the acoustic wave in the frequency range [f_k, f_{k + 1}], and this bit is 0 otherwise;</li><li>the last 8 bits of the code are determined from the time signal S (t) which is subdivided into 9 predetermined time slots [t_k, t_{k + 1}], k = 1..9: the bit of rank k + 8 is equal to 1 for example if the average value of the signal strength during the time interval [t_k, t_{k + 1}] is greater than the average signal strength during the time interval [t_{k + 1}, t_{k + 2}], for k = 1..8, and this bit is 0 otherwise.</li></ul>

0047In this particular example, the codes of the correspondence table would be determined during the learning phase, by calculating as indicated above the codes which correspond to the signals picked up by the acoustic sensor 6 when we generate impacts on the different active zones 10.

0048Furthermore, as shown in the <figref idref="f0002">Figures 4 and 5</figref>, it may be possible to use two acoustic sensors 6 (SENS.1 and SENS.2), in particular when the object 5 has symmetries such that there may be a risk of confusion between the signals coming from two active zones 10 different. If necessary, more than two acoustic sensors 6 could be used, although the preferred solutions use one or two sensors 6.

0049When two or more sensors are used, two choices are possible:<ol id="ol0002" compact="compact"><li>1) mixing of the signals from the various sensors and processing of the overall signal according to the method described above.</li><li>2) or, preferably, individual processing of the signals from the various sensors with the method described above and cross-checking of the results:<ul id="ul0012" list-style="dash" compact="compact"><li>if the active zones 10 determined from the different sensors have identical numbers, then it is determined that the zone which has received an impact is this,</li><li>in the other cases, one can either consider the signal received as being a false alarm, or determine the zone which received an impact for example by intercorrelation between the intercorrelation functions Ci (t) determined for each sensor, or by means more complex such as neural networks or others.</li></ul></li></ol>

0050It will be noted that the two acoustic sensors can be of different types and / or pick up different sizes and / or their signals can be processed differently to identify the active zones 10 receiving impacts. For example, one of the acoustic sensors can be used to record the signal S (t) received, while the other can only be used to determine a time difference between the arrival of the acoustic wave on the two sensors.

0051The second sensor could moreover not pick up the acoustic wave propagated in the solid object 5, but the acoustic wave propagated in the air during the impact.

0052As shown in the <figref idref="f0003">figure 6</figref>, the object forming an acoustic interface can be constituted by a computer screen 3 or a television screen to which the sensor 6 is fixed. The surface receiving the impacts can advantageously be the window 15 of the screen, which can allow in particular display on screen 3 the delimitation of the active zones 10 and their meaning. This variant could be used for example to program a video recorder, in particular in the case where the screen 3 would be a television screen (the central unit 2 would then be replaced by the video recorder).

0053As shown in the <figref idref="f0003">figure 7</figref>, the object forming an acoustic interface can also be constituted by a glass door 16 or the like. In the example shown in the<figref idref="f0003">figure 7</figref>, the surface 17 which carries the active areas 10 is constituted by the glass surface of the door, and, still in the particular example shown in this figure, the acoustic sensor 6 is fixed to a wooden part of the door 16.

0054In the example shown in the <figref idref="f0003">figure 8</figref>, the object forming an acoustic interface is a tablet 18 designed specifically to serve as an acoustic interface. This tablet may for example comprise a rigid frame 19 secured to a base 20 also rigid which carries the acoustic sensor 6.

0055A flexible membrane 22, made for example of elastomer, is stretched on the frame 19 a short distance above the bottom 21. This flexible membrane 22 is provided with rigid pins 23 under its underside (it may for example be glass half-spheres which are glued under the membrane 22). Thus, when a user taps on the membrane 22 and in particular on an active area 10 carried by this membrane, this action generates an impact of at least one pin 23 on the bottom 21 of the frame of the tablet 18. This variant has the the advantage of producing impacts which depend relatively little on the way in which the user taps on the membrane 22 (with the finger or the nail or a tool, with more or less force, etc.).

0056In the embodiments of <figref idref="f0003">figures 6 to 8</figref>, the process used can be identical or similar to that described above and make it possible to match an impact generated on the surface of the object forming an acoustic interface, either with an active area 10, or with no active area.

0057But it is also possible, in all the embodiments of the invention using several active surfaces (possibly punctual), to determine the position of the impact on the surface 9 of the object 5 forming an acoustic interface (see l example of the <figref idref="f0003">figure 9</figref>), even when this impact is not on one of the active zones. This gives a continuous or pseudo-continuous acoustic interface (allowing similar operation for example to a computer mouse, an optical pencil, a touch screen or the like).

0058In this case, during the recognition stage:<ul id="ul0013" list-style="dash" compact="compact"><li>resemblance values ​​representative of the resemblance between the received signal and the predetermined signals are determined (in particular values ​​resulting from the above-mentioned intercorrelation functions Ci (t), for example their maximum D (i) defined above),</li><li>the impact is associated with a number p at least equal to 2 of adjacent active zones corresponding to a maximum resemblance, called active reference zones R1-R4 (p may advantageously be worth 4 in particular for positioning the impact along two dimensions x, y, or where appropriate less than 4, in particular if the impact should only be positioned in one dimension x or y): one can for example first determine the zone R1 as being the active zone 10 having the maximum resemblance value D (i), then determine, among the active zones adjacent to R1, the three zones R2-R4 which give the most high of the resemblance value D (i));</li><li>then the position of the impact I on the surface 9 is determined as a function of the resemblance values ​​D (i) assigned to the active reference zones R1-R4.</li></ul>

0059During this last step, it is advantageous to determine the position of the impact on the surface so that the similarity values ​​assigned to the active reference zones correspond as closely as possible to theoretical similarity values ​​calculated for said active zones of reference for an impact generated in said position on the surface.

0060These theoretical similarity values ​​can in particular be functions of the position of the impact on the surface, determined in advance for each possible set of active reference zones.

0061The functions in question can be determined during the learning step, for example by adjusting a standard function on the resemblance values ​​of the active zones between them. The standard function in question may depend on the shape of the object and be determined in advance, either theoretically or experimentally.

0062To take a concrete example, the theoretical similarity function Rth (X, Y) between two points X, Y of the surface 9 can correspond to the maximum of the intercorrelation function between the signals Sx (t) and Sy (t) captured by the sensor 6 respectively when impacts are generated at these two points X, Y, and this function can for example be of the type Rth (X, Y) = (sin (a (β) .d)) / (a ​​(β ) .d), approximated for example by Rth (X, Y) = 1- [a (β) .d]²/ 6, where:<ul id="ul0014" list-style="dash" compact="compact"><li>d is the distance between X and Y,</li><li>β is an angle between for example the x axis (or the y axis) and the XY direction,</li><li>and a (β) is a coefficient depending on the angle β according to an elliptical function: <maths id="math0003"><math display="block"><mi mathvariant="normal">at</mi><mspace width="1em" /><mfenced><mi mathvariant="normal">β</mi></mfenced><mo mathvariant="normal">=</mo><mi mathvariant="normal">at</mi><mo>⁢</mo><mn mathvariant="normal">1.</mn><mrow><mi>cos</mi><mo mathvariant="normal">(</mo><mi mathvariant="normal">β</mi><mo mathvariant="normal">+</mo><mi mathvariant="normal">β</mi><mo>⁢</mo><mn mathvariant="normal">0</mn><mo mathvariant="normal">)</mo></mrow><mo mathvariant="normal">+</mo><mi mathvariant="normal">at</mi><mo>⁢</mo><mn mathvariant="normal">2.</mn><mo>⁢</mo><mi>sin</mi><mspace width="1em" /><mfenced><mi mathvariant="normal">β</mi><mo mathvariant="normal">+</mo><mi mathvariant="normal">β</mi><mo>⁢</mo><mn mathvariant="normal">0</mn></mfenced><mo mathvariant="normal">,</mo></math><img file="EP1512116B9_D0003.tif" /></maths></li></ul>where β0 is an angle representative of the orientation of the ellipse.

0063We can determine the function Rth for each possible set of active reference areas R1-R4, from predetermined signals Ri (t) from the signal library, captured by generating impacts respectively on these active reference areas during the learning phase.

0064For this purpose, for a given set of four reference zones R1-R4, it is possible to calculate the maximum of the intercorrelation function of the signal R1 (t) corresponding to R1, with each of the signals R2 (t), R3 (t ), R4 (t) corresponding to zones R2-R4. We deduce values ​​from a1, a2 and β0. We can then do the same from the reference areas R2, R3 and R4, which gives values ​​of a1, a2 and β0 each time, then take the average of the four values ​​thus found respectively for a1, a2 and β0: these average values ​​then determine the function Rth for the set of reference zones R1-R4. As a variant, the function Rth could be determined by an iterative optimization process (of the least squares method type) to minimize an error function between the theoretical resemblance function and the maxima of the intercorrelation functions between the signals R1 ( t), R2 (t), R3 (t) and R4 (t) taken two by two.

0065Once the aforementioned theoretical similarity functions Rth have been determined, when we seek to determine the position of an impact I between four adjacent active zones R1-R4 (advantageously punctual), this position can for example be determined by an iterative optimization process by minimizing an error function between the values ​​D (i) defined previously (D (i) = Max (Ci (t)) i being here the number of the active reference area Ri considered) and the theoretical similarity values ​​Rth (I, Ri). For example, we can minimize an error function E equal to the sum of the values ​​(D (i) -Rth (I, Ri))².

0066The process which has just been described is of course not limited to the examples which have just been described; it has many applications, among which:<ul id="ul0015" list-style="dash" compact="compact"><li>the use of windows or other surfaces as an input interface 4, in shops, museums, art galleries, or the like to allow customers or visitors to be presented with details on a screen or by means of a loudspeaker concerning the products or works exhibited,</li><li>the use of windows or other surfaces of display panels as input interfaces 4, allowing passers-by to be presented, for example, with details of the advertisements being displayed, or else to be presented with general information concerning a municipality or another place (for example, news or practical information, for example a map of the place), or others, these details or information being presented for example on a screen visible at the bottom of the display panel,</li><li>the use of parts of walls, of the floor, or of any other object as an input interface 4 for example for controlling home automation systems (it is thus notably possible to allow the inhabitants of an apartment to determine for themselves the locations of the switches , consisting simply of the active areas 10 above, positioned on the walls or the like at the desired locations),</li><li>the use of parts of walls, of the ground, or of any other object as an input interface 4 for example for controlling industrial machines in particular in a hostile environment (places containing explosives, places with high temperature, places with high radioactivity, etc. .),</li><li>the use of smooth and easy-to-maintain surfaces as an input interface 4, to constitute input keyboards for household objects such as a refrigerator, washing machine or the like,</li><li>the use of building door panels as input interfaces 4, constituting for example virtual keypads of digital code,</li><li>the use of the ground to locate the position of a person walking on it,</li><li>the production of keypads or control panels insensitive to pollution, bad weather or other external aggressions, in industrial, military or even domestic applications (the acoustic sensor (s) may possibly be completely integrated into the object which serves as an interface for entry, especially if it is an object at least partially molded in plastic); when these input interfaces must control a device (for example a microcomputer) comprising a display screen, the keyboard or acoustic control panel can be formed by the screen itself or by a transparent wall covering this screen.</li><li>the creation of input interfaces in automobiles or other vehicles.</li></ul>

0067Note also that the input interface 4 described above could be provided with processing means making it possible to locally recognize the acoustic signals S (t) coming from the active zones 10, the input interface 4 then sending directly to the central processing unit 2, or to any other user electronic device, only coded signals directly indicating which active zone 10 has been touched by the user and, where appropriate, information relating to the impact: force of the impact and nature of the impact.

0068It will be noted that the method according to the invention does not require that the object 5 has a homogeneous or predetermined structure, or be produced with particular care, or be produced with very precise dimensions, or with specific surface states. On the contrary, the more heterogeneous and / or irregular the object 5, the more the acoustic signals emitted by the different active zones 10 will differ from one another, and the better the recognition of the acoustic signals will be. One can even in certain cases voluntarily create heterogeneity such as cavities or others in the object 5 to facilitate the recognition of the acoustic signals coming from the active zones 10.

0069Furthermore, when the predetermined signals from the signal library are determined during a learning phase, it is possible to use a piezoelectric sensor connected by all means. known means to the central unit 2 and fixed either on the user's finger, or on the object (stylus or other) used to generate impacts on the active areas of the object 5. In this case, the pulse signal generated by the piezoelectric sensor during each impact can be used to trigger the acquisition of the predetermined acoustic signal intended to supply the signal library, and / or to measure the intensity of the impact , this intensity measurement can be used for example to invalidate certain acquisitions of predetermined signals, in particular when the intensity is less than a predetermined threshold or when this intensity is not included in a predefined interval.

0070Furthermore, when the predetermined signals from the signal library are determined during a learning phase, it may be advantageous to retain only the acoustic signals picked up whose amplitude is greater than a first relatively high reference threshold. . In this case, during normal operation of the device, one can then take into account the acoustic signals whose amplitude exceeds a second predetermined threshold significantly lower than the first threshold. The first predetermined threshold can thus be equal to several times (at least two to three times) the time average value of the absolute amplitude of the ambient noise, measured for example over a few seconds, while the second predetermined threshold can for example be equal at 1.5 times this average value. In this way, only good quality reference signals are recorded during the learning phase, while retaining a high sensitivity of the device during its normal operation.

0071If necessary, the central unit 2 can be provided with an auxiliary programming keyboard (not shown) which can be used in particular during the learning phase, to indicate for example what type of signal is generated. The type of signal generated can in particular be one of the following types:<ul id="ul0016" list-style="dash" compact="compact"><li>new signal to replace one of the reference signals from the signal library (the identification of the replaced reference signal can thus be communicated to the central unit 2 by means of the auxiliary keyboard),</li><li>new reference signal (either for a preexisting but incomplete reference library, or for a new reference library corresponding in particular to new conditions of temperature, humidity or state of the object 5),</li><li>new signal intended to check a reference signal already existing in a library of signals.</li></ul>

0072Furthermore, when the predetermined signals of the signal library are determined during a learning phase, it can be provided, if necessary, to validate the reference signals of this library only when they are confirmed by generation of one or more impacts on the same active area, within a predetermined period of time following the generation of a first impact.

0073When the predetermined signals of the signal library are determined during a learning phase, the impacts generated on the object 5 during this learning phase can be generated:<ul id="ul0017" list-style="dash" compact="compact"><li>either with a hard object such as a stylus, in which case the same stylus will preferably be used during normal operation of the device,</li><li>either with a more damping object such as for example a hard plastic eraser attached to the end of a pen or the like (the inventors have thus been able to obtain good results with a hard plastic eraser for transparencies of the "Staedler" brand), to which case the impacts on the object 5 can then be generated as well with relatively hard objects as with less hard objects (fingernail, finger pulp or other) during the normal operation of the device.</li></ul>

0074Furthermore, as a variant of the method described above for recognizing the active area 10 from which the received signal S (t) comes, it is possible to use the following method:<ol id="ol0003" compact="compact"><li>(1) during the learning phase, the Fourier transform R is calculated_i(ω) of each acoustic signal R_i(t) generated by an impact on the active area i, where i is an index between 1 and n: <maths id="math0004"><math display="block"><msub><mi mathvariant="normal">R</mi><mi mathvariant="normal">i</mi></msub><mspace width="1em" /><mfenced><mi mathvariant="normal">ω</mi></mfenced><mo mathvariant="normal">=</mo><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">R</mi><mi mathvariant="normal">i</mi></msub><mfenced><mi mathvariant="normal">ω</mi></mfenced><mo mathvariant="normal">|</mo><mn mathvariant="normal">.</mn><msup><mi mathvariant="normal">e</mi><mrow><mi mathvariant="normal">i ϕ i</mi><mspace width="1em" /><mfenced><mi mathvariant="normal">ω</mi></mfenced></mrow></msup></mrow></math><img file="EP1512116B9_D0004.tif" /></maths>We only keep this Fourier transform of the phase component, in the only frequency bands ω where the amplitude of the spectrum is greater than a predetermined threshold. The frequency form of the stored reference signal is therefore expressed in the form R '_i(ω) = e^{j ϕi (ω)} for frequencies ω at which | R_i(ω) | is greater than the predetermined threshold, and R '_i (ω) = 0 at other frequencies ω. The predetermined threshold in question may for example be equal to the maximum of MAX / D and of | B (ω) |, where:<ul id="ul0018" list-style="dash" compact="compact"><li>MAX can be either the maximum value of | R_i(ω) |, ie the maximum value of the modules | R_i(ω) | each standardized in energy, ie the maximum value of the envelope of the mean of the modules | R_i(ω) | each standardized in energy,</li><li>D is a constant, for example equal to 100,</li><li>IB (w) l 1 is the average of several noise spectra in object 5, acquired at different times.</li></ul></li><li>(2) During normal operation of the device, each signal received S (t) undergoes the same processing as in step (1) above, so that a signal S '(ω) = e is obtained.^{jψ (ω)} for frequencies ω at which | S (ω) | is greater than the aforementioned predetermined threshold, S '(ω) being equal to 0 at the other frequencies.</li><li>(3) We then calculate a product P_i(ω) equal to S '(ω) multiplied by the conjugate of R' (ω) for i = 1 ... n.</li><li>(4) We normalize the products P_i(ω) by dividing them by their integrals.</li><li>(5) Then perform the inverse Fourier transform of all the products P_i(ω), and we obtain temporal functions X_i(t).</li></ol>

0075According to the different functions X_i(t), and in particular according to their maximum, it is then possible to assign the signal S (t) to one of the active zones 10. By way of example, the maximum value (in relative value or in absolute value), of the different functions X_i(t), and assign the signal S (t) to the active area i which corresponds to the function X_i(t) with the greatest maximum. Optionally, we can also compare the maximum of the function X_i(t) retained with a threshold defined in advance, for example equal to 0.6, and decide that the signal S (t) must be allocated to the area i when the maximum of X_i(t) is greater than this threshold (if several functions X_i(t) have their maximum greater than 0.6, then we keep only the function X_i(t) of maximum maximum).

0076It is possibly possible to verify that the assignment of the signal S (t) to the active area i is correct, for example by calculating a value MM_i = M_i/ M where M_i is the maximum of the absolute value of X_i (t) and M is the average value of all M values_i. The assignment of the signal S (t) to the active area i can then be considered to be correct if this value MM_i is greater than a certain limit, for example equal to 1.414.

0077Note also that the MM values_i above can be calculated by replacing S '(ω) with R'_i(ω), so as to obtain information on the spatial resolution of the active areas. In particular, we can thus verify that an active area of ​​index i is not likely to be confused with another, by verifying that the value MM_i corresponding is greater than a predetermined limit, for example greater than 1.414.

0078Furthermore, it is also possible to take into account different ambient parameters (temperature, hygrometry, mechanical stresses, etc.) by modifying the predetermined signals of the signal library as a function of the ambient parameter or parameters.

0079One of the following correction methods can be used for this purpose:<ul id="ul0019" list-style="dash" compact="compact"><li>linear temporal dilation or contraction of the reference signals from the signal library: in this case, the reference signals R_i(t) of the signal library are replaced by R signals_i(αt), where α is a non-zero positive multiplying coefficient which is a function of the ambient parameters, this coefficient α being able to be determined theoretically, or even experimentally for a given material, or even experimentally for each object 5;</li><li>linear expansion or contraction of the captured signals S (t): in this case, the reference signals R_i(t) are left unchanged, but the received signal S (t) is replaced by S (αt) where α is a coefficient as defined above;</li><li>non-linear expansion or contraction in frequency of the reference signals: in this case, we replace the frequency signals R '_i(ω) by R '_i(ω '), with <maths id="math0005"><math display="inline"><mi>ωʹ</mi><mo>=</mo><mfrac><mrow><mi>ω</mi><mo>⁢</mo><msup><mi>β</mi><mn>2</mn></msup></mrow><mrow><mn>1</mn><mo>+</mo><msqrt><mrow><mo>(</mo><mi>ω</mi><mo>/</mo><msub><mi>ω</mi><mi>NOT</mi></msub><mo>)</mo><mn>.</mn><mfenced><mi>β</mi><mo>-</mo><mn>1</mn></mfenced></mrow></msqrt></mrow></mfrac><mo>,</mo></math><img file="EP1512116B9_D0005.tif" /></maths>where ω_NOT is equal to half the sampling frequency of the processing device, and β is a coefficient determined theoretically or experimentally;</li><li>Nonlinear frequency dilation or contraction of the received signal S (t): in this case, the reference signals from the signal library are left unchanged, and the signal S '(ω) is replaced by S' (ω '), étant 'being defined above.</li></ul>

0080In the two aforementioned cases of non-linear frequency expansion or contraction, it is also possible to use an averaged phase correction, in which case the signals R_i (ω ') are replaced by R_i (ω '). M' (ω) / N '(ω) or the signals S (ω') are replaced by S (ω '). M' (ω) / N '(ω). In either of these formulas, N '(ω) = M (ω) / | M (ω) |, and N' (ω) = N (ω) / | N (ω) |, M (ω) being equal to the average of all R_i(ω) and N (ω) being equal to the average of all R_i(ω ').

0081The various abovementioned corrections of the reference signals R_i(ω) or of the signal S (ω) can be carried out either automatically by the central unit 2, in particular as a function of information given by one or more sensors (not shown), or manually by the user.

0082Furthermore, it will be noted that the central processing unit 2 can include several libraries of reference signals adapted to different values ​​of the ambient parameters.

0083Furthermore, to adapt to the types of impact generated during the use of the device, and in particular to adapt to the use of either a finger of the user, or another object to generate the impacts, it may be advantageous to ask the user to generate impacts on one or more predetermined active zones 10, for example two active zones of indices m and p. We thus capture two time signals S_m(t) and S_P(t), whose Fourier transforms S are calculated_m(ω) and S_p(ω), then we calculate the mean M₁(ω) of the following two terms:<ul id="ul0020" list-style="dash" compact="compact"><li><maths id="math0006"><math display="inline"><mrow><mo>(</mo><msub><mi mathvariant="normal">R</mi><mi mathvariant="normal">m</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo><mn>.</mn><mrow><mo>|</mo><msub><mi mathvariant="normal">S</mi><mi mathvariant="normal">m</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo><mo>|</mo><mo>)</mo><mo>/</mo><mrow><mo>(</mo><mrow><mo>|</mo><msub><mi mathvariant="normal">R</mi><mi mathvariant="normal">m</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo><mo>|</mo><mn>.</mn><msub><mi mathvariant="normal">S</mi><mi mathvariant="normal">m</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo></mrow><mo>)</mo><mo>,</mo></mrow></mrow></mrow></math><img file="EP1512116B9_D0006.tif" /></maths></li><li>and <maths id="math0007"><math display="inline"><mrow><mo>(</mo><msub><mi mathvariant="normal">R</mi><mi mathvariant="normal">p</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo><mn>.</mn><mrow><mo>|</mo><msub><mi mathvariant="normal">S</mi><mi mathvariant="normal">p</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo><mo>|</mo><mo>)</mo><mo>/</mo><mrow><mo>(</mo><mrow><mo>|</mo><msub><mi mathvariant="normal">R</mi><mi mathvariant="normal">p</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo><mo>|</mo><mn>.</mn><msub><mi mathvariant="normal">S</mi><mi mathvariant="normal">p</mi></msub><mo>(</mo><mi mathvariant="normal">ω</mi><mo>)</mo></mrow><mo>)</mo><mn>.</mn></mrow></mrow></mrow></math><img file="EP1512116B9_D0007.tif" /></maths></li></ul>This average M₁(ω) is then used in step (3) defined above to replace the product P_i(ω) by M₁(ω) .P_i(ω), this product then being used in place of P_i(ω) in step (4).

0084Furthermore, it will be noted that the invention allows a user to define active zones as he pleases, and the central unit 2 can be adapted to keep this definition of active zones active only during the effective use of the object 5 as an acoustic interface. In this case, the above-mentioned definition of the active zones is erased by the central unit 2 after a certain period of non-use of the device.

0085It will also be noted that the function generated by an impact on an active area can be modulated if necessary as a function of the intensity of this impact.

0086It will also be noted that, when the object 5 presents resonance phenomena which cause prolonged propagation of acoustic waves at each impact on the active areas, it may be advantageous to raise the detection threshold of the acoustic signals S (t) ( for example up to 0.5 times the maximum value admissible by the electronic signal acquisition system S (t)) when a signal S (t) has been detected, then lowering this detection threshold (especially exponentially) to its normal level: thus, multiple detections of the same impact are avoided.

0087It will be noted that, in all the embodiments of the invention, it would possibly be possible to define a single active area on the object 5, in which case it is nevertheless possible to code several functions on this unique active area, for example following the number of impacts generated consecutively on the same area.

0088Furthermore, the active zones 10 may possibly not be defined in advance, but simply be defined as a function of the successive impacts received during the use of the device. Thus, the device can for example be designed to include three active zones, which are simply defined each on receipt of a first impact on each zone, and which are then recognized as "first zone", "second zone", and " third zone ", upon receipt of the following impacts.

0089Furthermore, when there are a large number of active zones, it is possible, if necessary, to use an automated device to generate the reference signals stored in the signal library during the learning phase. This automated device could for example comprise a two-dimensional displacement system, comprising two stepping motors, for example for moving an excitation stylus or the like on the surface of the object 5 and for generating impacts by means of this stylus, actuated for example by a solenoid, at the level of the various active zones.

0090Still in the case where the active zones 10 are very numerous, it may be possible to divide them into several similarity groups. In this case, during the current use of the device, when an impact generates a signal S (t), a first processing makes it possible to attach this signal S (t) to one of the resemblance groups, then a refined processing allows this signal S (t) to be assigned to one of the active zones of this resemblance group.

0091It will also be noted that the same central unit 2 could, if necessary, process the signals coming from several objects 5. In addition, it is also possible to directly interface the acoustic sensor or sensors with a network, in particular an IP network, so as to direct the signals received to a unique IP address from which these signals can be processed by any computer connected to the IP network.