Method and apparatus for haptic vibration response profiling and feedback

RELATED APPLICATIONS

0001This application claims priority to provisional U.S. Patent Application Ser. No. 61/470,764, entitled “Touch Screen and Haptic Control” filed on Apr., 1, 2011, the content of which is incorporated herein in its entirety.

BACKGROUND

0002The present invention relates to user interface control and haptics generation.

0003Haptics refers to the sense of touch. In electronic devices, haptics relates to providing a touch sensory feedback to the user. Electronic devices incorporating haptics may include cell phones, PDAs, gaming devices, etc. The user interacts with electronic devices through a user interface, such as a touch screen; however, the user often does not know if the user's desired function was recognized or is being performed by the electronic device. Thus, electronic devices generate a haptic feedback in the form of a vibro-tactile sensation (e.g. a simulated “click”) to alert the user of the electronic device's performance. Stated differently, haptic feedback lets the user know what is going on with the electronic device. In a gaming electronic device, for example, haptics can provide a sensory stimuli according to game interactions.

0004Haptic feedback can be generated by electro-mechanical systems. An electrical system produces a drive signal that will then cause a mechanical system to produce the haptic effect. For example, an actuator incorporating a moving mass can be used to generate haptic effects. A linear resonant actuator (LRA) is an example of one such actuator in which a moving mass is spring loaded. For efficient haptic generation using an LRA, the spring loaded mass may be driven at its mechanical resonant frequency, which is the natural vibration frequency of the spring loaded mass. Also, the magnitude of the haptic effect may be controlled by the amplitude of the actuator driving signal.

0005In some conventional systems, BEMF (Back Electromotive Force) is used to tune the actuator. BEMF is an electrical signal that is induced into a the electrical connections of the motor by the movement of a permanent magnet (which as a mass) relative to a stationary wire wound coil. In theory, the BEMF signal will have electrical properties (e.g., frequency, amplitude) that correspond to the mechanical vibrations in the actuator. However, the BEMF measurements may be unreliable at times because of electrical interferences in the system and, thus, may not provide accurate representation of the vibrations.

0006Hence, the inventors recognized a need in the art for an adaptive haptic effect generation technique based on accurate representations of the mechanical vibrations.

BRIEF DESCRIPTION OF THE DRAWINGS

0007<figref idref="DRAWINGS">FIG. 1(</figref><i>a</i>) is a simplified block diagram of a haptic generation system according to an embodiment of the present invention.

0008<figref idref="DRAWINGS">FIG. 1(</figref><i>b</i>) is a simplified block component diagram of haptic generation system according to an embodiment of the present invention.

0009<figref idref="DRAWINGS">FIG. 2</figref> is a simplified block diagram of a user interface (UI) controller according to an embodiment of the present invention.

0010<figref idref="DRAWINGS">FIG. 3</figref> illustrates exemplary haptic profile representations according to an embodiment of the present invention.

0011<figref idref="DRAWINGS">FIGS. 4(</figref><i>a</i>)-<b>4</b>(<i>c</i>) illustrate haptic actuator configurations according to embodiments of the present invention.

0012<figref idref="DRAWINGS">FIG. 5</figref> illustrates an interference pattern according to an embodiment of the present invention.

0013<figref idref="DRAWINGS">FIG. 6</figref> is a simplified block diagram of feedback generation system according to an embodiment of the present invention.

DETAILED DESCRIPTION

0014Embodiments of the present invention may provide a device to adaptively generate a haptic effect. The device may include a controller to generate a haptic command associated with a haptic profile and a haptic driver to generate a drive signal based on the haptic command, wherein the drive signal causes an actuator to produce vibrations corresponding to a haptic effect. Further, the device may include a sensor, coupled mechanically to the actuator, to measure the vibrations. The controller may adjust the haptic command according to a property of the measured vibrations. Therefore, the device may continuously tune haptic effect generation according to vibration measurements.

0015<figref idref="DRAWINGS">FIG. 1(</figref><i>a</i>) is a simplified block diagram and <figref idref="DRAWINGS">FIG. 1(</figref><i>b</i>) is a simplified component block diagram of an adaptive haptic generation system <b>100</b> according to an embodiment of the present invention. The system <b>100</b> may include a User Interface (UI) controller <b>110</b>, a haptics driver <b>120</b>, a haptics actuator <b>130</b>, a mechanical connector <b>140</b>, a mechanical sensor <b>150</b>, and a housing <b>160</b>. The system <b>100</b> may be provided within a consumer electronic device such as a cell phone, PDA, gaming device, etc.

0016The UI controller <b>110</b> may be coupled to the haptics driver <b>120</b> and a mechanical sensor <b>150</b>. The UI controller <b>110</b> may generate a control signal based on a desired haptic effect and may transmit the control signal to the haptics driver <b>120</b>. For example, a user may select an icon on a touchscreen, and the UI controller <b>110</b> may generate a control signal corresponding to a desired haptic effect such as a clicking vibration to provide a feedback stimuli.

0017<figref idref="DRAWINGS">FIG. 2</figref> is a block diagram of a UI controller <b>110</b> from <figref idref="DRAWINGS">FIG. 1</figref> according to an embodiment of the present invention. The UI controller <b>110</b> may include a processor <b>210</b>, a memory <b>220</b>, a serial interface <b>230</b>, and a general purpose input/output (GPIO) <b>240</b>.

0018The processor <b>210</b> may control the operations of the UI controller <b>110</b> according to instructions saved in the memory <b>220</b>. The memory <b>220</b> may also store sensor data from the coupled mechanical sensor <b>150</b>, and the memory <b>220</b> may store haptic effect profiles as will be described below. The memory <b>220</b> may be provided as a non-volatile memory, a volatile memory such as random access memory (RAM), or a combination thereof.

0019The serial interface <b>230</b> may connect to the coupled mechanical sensor <b>150</b>. The serial interface <b>230</b>, for example, may be provided as high speed <b>12</b>C interfaces. The GPIO <b>240</b> may connect to the haptics driver <b>120</b>. The UI controller <b>110</b> may also be coupled to a host system (not shown) of the device via the GPIO <b>240</b>.

0020Returning to <figref idref="DRAWINGS">FIG. 1</figref>, the haptics driver <b>120</b> may receive the control signal from the UI controller <b>110</b> and may generate a drive signal based on the control signal. The drive signal may be an analog signal. The drive signal may be a current signal or a voltage signal.

0021The haptics driver <b>120</b> may be coupled to the haptics actuator <b>130</b>. The haptics actuator may be embodied as piezoelectric elements, linear resonant actuators (LRAs), eccentric rotating mass actuators (ERMs), and/or other known actuator types. The haptics driver <b>120</b> may transmit the drive signal to the haptics actuator <b>130</b> causing it to vibrate according to the drive signal properties. The vibrations may be felt by the user providing a vibro-tactile sensory feedback stimuli.

0022In an embodiment, the haptics actuator <b>130</b> may include a mechanical system such as a motor that vibrates to generate the desired haptic effect. For example, the haptics actuator <b>130</b> may include a coil motor with a spring loaded mass and a permanent magnet. The coil motor may cause the spring loaded mass to vibrate to generate the haptic effect. The haptics actuator <b>130</b> may also include magnetic coils to generate the motion.

0023The vibrations may also be captured by the mechanical sensor <b>150</b> via the mechanical connector <b>140</b>. The mechanical connector <b>140</b> may be provided as a printed circuit board (PCB) where other components, including the UI controller <b>110</b>, the haptics driver <b>120</b>, the haptics actuator <b>130</b>, and/or the mechanical sensor <b>150</b>, may be mounted. The mechanical sensor <b>150</b> may be provided as an accelerometer sensor. For example, the mechanical sensor <b>150</b> may be provided as a Microelectromechanical Systems (MEMS) accelerometer sensor. Further, the MEMS accelerometer may be provided as a single axis or multi-axis sensor(s) to measure force(s) corresponding to the vibrations in the haptics actuator <b>130</b>.

0024The mechanical sensor <b>150</b> may measure mechanical properties of the generated haptic effect vibrations, and may transmit the measured mechanical properties to the UI controller <b>110</b>. In response to the measured mechanical properties, the UI controller <b>110</b> may adjust the control signal for the respective haptic effect.

0025In operation, the UI controller <b>110</b>, responsive to a user action, may generate a control signal corresponding to a particular haptic effect. The control signal may be a haptic effect command. The UI controller <b>110</b> may generate the control signal based on a stored haptic effect profile associated with the desired haptic effect.

0026<figref idref="DRAWINGS">FIG. 1(</figref><i>b</i>) is a simplified component block diagram of the adaptive haptic generation system <b>100</b> according to an embodiment of the present invention. The UI controller <b>110</b>, haptics driver <b>120</b>, and mechanical sensor <b>150</b> may provided as circuit components that are mounted on a printed circuit board (PCB). The PCB and its mounted circuit components may be provided inside a housing <b>160</b>. The UI controller <b>110</b> may be electrically coupled to the haptics driver <b>120</b> and the mechanical sensor, for example via electrical traces on the PCB. The haptics driver <b>120</b> may also be electrically coupled to the haptics actuator <b>130</b>.

0027The haptics actuator <b>130</b> may include a mechanical system such as a motor that vibrates to generate the desired haptic effect. For example, the haptics actuator <b>130</b> may include a coil motor with a spring loaded mass and a permanent magnet. The coil motor may cause the spring loaded mass to vibrate to generate the haptic effect. The haptics actuator may also include magnetic coils to generate the motion. In an embodiment, the haptics actuator <b>130</b> may be connected to the housing <b>160</b>. Hence, when the haptic actuator <b>130</b> is driven and vibrates, the connected housing <b>160</b> may also vibrate, which may be felt by a user.

0028The haptics actuator <b>130</b> may be coupled to the mechanical sensor <b>150</b> via the mechanical connector <b>140</b>. In an embodiment, the mechanical connector <b>140</b> may be provided as the printed circuit board PCB on which other components, including the UI controller <b>110</b>, the haptics driver <b>120</b>, the haptics actuator <b>130</b>, and/or the mechanical sensor <b>150</b>, may be mounted. The mechanical sensor <b>150</b> may be provided as an accelerometer sensor. For example, the mechanical sensor <b>150</b> may be provided as a Microelectromechanical Systems (MEMS) accelerometer sensor. Further, the MEMS accelerometer may be provided as a single axis or multi-axis sensor(s) to measure force(s) corresponding to the vibrations in the haptics actuator <b>130</b>.

0029The mechanical sensor <b>150</b> may measure the vibrations (i.e., haptic effect) generated inside the housing <b>160</b> by the haptics actuator <b>130</b>. The mechanical sensor <b>150</b> may measure mechanical properties of the generated haptic effect vibrations, and may transmit the measured mechanical properties to the UI controller <b>110</b>. In response to the measured mechanical properties, the UI controller <b>110</b> may adjust the control signal for the respective haptic effect.

0030<figref idref="DRAWINGS">FIG. 3</figref> illustrates an exemplary representation of haptic effect profiles according to an embodiment of the present invention. The haptic effect profiles may be stored in the memory <b>220</b> of the UI controller <b>110</b> of <figref idref="DRAWINGS">FIGS. 1 and 2</figref>. The memory <b>220</b> may store a plurality of haptic effect profiles <b>310</b>, <b>340</b>, <b>350</b> and each haptic effect profile may correspond to a unique haptic effect.

0031Each haptic effect profile, for example Haptic effect <b>1</b> profile <b>310</b>, may include two sets of information, ideal properties <b>320</b> and their corresponding command variables <b>330</b>. In an embodiment of the present invention, the ideal properties <b>320</b> may be provided as read-only information with respect to the UI controller <b>110</b>. For example, the ideal properties <b>320</b> may be pre-programmed into the UI controller <b>110</b> or may be programmed by the host system.

0032The ideal properties <b>320</b> may include values corresponding to various characteristic of the respective haptic effect. For example, the ideal properties <b>320</b> may include an ideal frequency value, which may correspond to a desired resonant frequency of the haptic effect. The ideal properties <b>320</b> may include an ideal amplitude value, which may correspond to a desired magnitude (i.e., “volume”) of the haptic effect. The ideal properties may include an ideal duration value, which may correspond to a desired length of the haptic effect. The ideal properties may include an ideal start value, which may correspond to a desired initial vibratory feature of the haptic effect. For example, some haptic effects may have a ramping feature that gradually increases in magnitude over the duration of the haptic effect. The ideal properties may include an ideal braking value, which may correspond to how the haptic effect is stopped. For example, some haptic effects may use a braking technique to slow the vibration using the actuator's natural frequency rather than using an abrupt stop.

0033Each ideal property value may have a corresponding command variable <b>330</b>. The command variables <b>330</b> may be associated with the control signal that will effectuate the respective haptic effect and, consequently, the desired haptic effect properties. In an embodiment, the UI controller <b>110</b> may generate a control signal (i.e., haptic command) using the command variables <b>330</b> that will correspond to a drive signal that will generate the desired haptic effect.

0034For example, a frequency factor may be haptic command variable that causes the haptic actuator <b>130</b> to vibrate at an ideal resonant frequency for that respective haptic effect. An amplitude factor may be a haptic command variable that causes the haptic actuator <b>130</b> to vibrate at an ideal magnitude for that respective haptic effect. A duration factor may be a haptic command variable that causes the haptic actuator <b>130</b> to vibrate for an ideal length for that respective haptic effect. A start factor may be a haptic command variable that causes the haptic actuator <b>130</b> to begin vibrating in a certain manner for the initial phase of that respective haptic effect. A braking factor may be a haptic command variable that causes the haptic actuator <b>130</b> to stop vibrating in a certain manner for that respective haptic effect.

0035Ideally, the vibrations of the haptic effect would display the associated ideal properties <b>320</b> of the command variables <b>330</b>. However, various factors such as manufacturing irregularities or wear and tear of parts may lead to discrepancies between the ideal properties and actual properties of the generated haptic effect. Thus, haptic effect generation may be adaptively tuned according to an embodiment of the present invention.

0036Consequently, the command variables <b>330</b> may be programmable by the UI controller <b>110</b>. The command variables <b>330</b> may be adjusted continuously, dynamically, and/or iteratively. According to an embodiment of the present invention, the UI controller <b>110</b> may adjust the command variables <b>330</b> based on measurement values received from the mechanical sensor <b>150</b>. The adjustments may synchronize the actual measured properties towards the ideal properties <b>320</b> of the respective haptic effect. Thus, the UI controller <b>110</b> may compare the received measurement values to the stored ideal properties <b>320</b>, and based on the comparison(s), the UI controller <b>110</b> may adjust the command variables <b>330</b> accordingly.

0037In an embodiment of the present invention, the adjustments may be applied to the current haptic effect from which the measurements were procured. In other words, the UI controller <b>110</b> and haptics driver <b>120</b> may cause the haptics actuator <b>130</b> to vibrate to generate the desired haptic effect. The mechanical sensor <b>150</b> may measure the mechanical properties of the haptic effect and may report the measured values to the UI controller <b>110</b>. Based on the comparison(s) between the measurement values and ideal properties, the UI controller <b>110</b> may adjust the haptic command while the current haptic effect is still occurring. The haptics driver <b>120</b>, in turn, may drive the haptic actuator <b>130</b> according to the adjusted haptic command, which will adjust the current haptic effect before it has finished.

0038In another embodiment of the present invention, the adjustments may applied to a subsequent haptic effect request. Thus, the adjustments may be saved, and the next time that haptic effect is requested, the UI controller <b>110</b> may generate the haptic command based on the saved adjustments.

0039In an embodiment of the present invention, the command variables <b>330</b> may be provided as look up tables (LUTs). The current factors for the command variables <b>330</b> may be indicated by a pointer, which may be incremented/decremented based on the comparison(s) between the measurement values and ideal properties.

0040Haptic elements may be provided at different locations inside the electronic device. The haptic effect may vary depending on the location of the haptic elements. <figref idref="DRAWINGS">FIG. 4</figref> illustrates configurations <b>400</b> of piezoelectric actuators for use in accordance with embodiments of the present invention. As illustrated in <figref idref="DRAWINGS">FIG. 4(</figref><i>a</i>), piezoelectric elements <b>420</b>.<b>1</b> may be affixed to a bottom surface of a touch screen <b>410</b>. In this embodiment, the piezoelectric elements <b>420</b>.<b>1</b> may be integrated with the touch screen <b>410</b> and may be provided at a layer above certain optically active elements of the screen (for example, higher than the LCD panel and backlight). For example, the piezoelectric elements <b>420</b>.<b>1</b> may be affixed to a glass layer of the screen, which typically is the layer that a user touches directly when interfacing with the touch screen <b>410</b>. In this embodiment, the piezoelectric elements may be provided at locations toward a periphery of the glass layer, outside the viewable area of the touch screen <b>410</b>. Electrical activation of the piezoelectric elements may cause flexion in a direction normal to the touch screen surface (up and down).

0041As illustrated in <figref idref="DRAWINGS">FIG. 4(</figref><i>b</i>), piezoelectric elements <b>420</b>.<b>2</b> may be affixed to a periphery of a touch screen <b>410</b>. Electrical activation of the piezoelectric element may cause flexion in a direction parallel to the touch screen <b>410</b> (laterally in the X and/or Y direction). This embodiment places the piezoelectric elements <b>220</b>.<b>2</b> at locations that are off the viewable area of the touch screen <b>410</b>.

0042As illustrated in <figref idref="DRAWINGS">FIG. 4(</figref><i>c</i>), piezoelectric elements <b>420</b>.<b>3</b> may be provided under a touch screen <b>410</b>. In this embodiment, the piezoelectric elements may be provided below the touch screen device <b>410</b> under the optically active elements of the screen (e.g. below the LCD, backlight, etc.). In this case, the piezoelectric elements <b>420</b>.<b>1</b> may be located anywhere along the lower surface of the screen. Electrical activation of the piezoelectric elements <b>420</b>.<b>2</b> may cause flexion in a direction normal to the touch screen surface (up and down). Although piezoelectric elements are illustrated above, embodiments of the present invention also permit use of LRAs or ERMs.

0043In an embodiment, a haptics driver may assert drive signals to the haptics actuators in a manner to create deflection waves laterally across a surface of the touch screen. Consider the configuration shown in <figref idref="DRAWINGS">FIG. 4(</figref><i>a</i>). In this configuration, haptics elements are provided at edges of the touch screen in four or, optionally, six locations. Each of the elements may be driven with a periodic activation pulse that causes deflection wave to radiate laterally across a screen surface away from the respective elements. Owing to the inertia of the touch screen, digital activation pulses may generate waves on the touch screen surface that are approximately sinusoidal. In such an embodiment, a haptics driver may control a frequency and phase of drive signals to individual haptics elements to tailor deflection waves that emanate from each actuator. Through constructive and destructive interference of these waves, the haptics driver may control the shape, orientation and magnitude of haptics effects.

0044<figref idref="DRAWINGS">FIG. 5</figref> is a graph illustrating a simulated interference pattern that may be generated across a surface of a touch screen according to an embodiment of the present invention. In this example, wave <b>2</b> has a frequency that is five times higher than wave <b>1</b>. As illustrated, the waves constructively interfere to cause a spike at a central position of the graph. The position of the spike may be shifted laterally within the graph by manipulating the phase of each wave. The shape of the total effect may be controlled by manipulating the frequency of each wave and the numbers of waves radiated onto the surface.

0045During operation, a haptics driver may control the frequency and phase of waves surface waves generated on a touch screen surface. In an embodiment, the frequency and phase may be adjusted based on mechanical sensor feedback measurements as described herein. To generate a spatially narrow sharp effect, waves may be controlled to provide a relatively large spike. The generate a spatially broader effect, waves may be controlled to provide a wider spike. Further, during operation, as a user's finger moves across the surface of the touch screen, the haptics driver may manipulate phase of the surface waves output to the touch screen in order to cause the waves to track the user's finger. Moreover, a haptics driver may convey a feeling of texture by manipulating both phase and frequency. In an application where a textured surface is displayed by the device (for example, in a gaming application), a haptic texture effect may be rendered as a user's finger slides along the displayed textured surface.

0046Principles of this invention may also find application in other adaptive sensory feedback generation. For example, a haptic effect may be accompanied with an auditory effect and/or a visual effect to provide a multi-sensory feedback stimulation. <figref idref="DRAWINGS">FIG. 6</figref> is a simplified block diagram of an adaptive multi-sensory feedback system <b>600</b> according to an embodiment of the present invention. The system <b>600</b> may include a User Interface (UI) controller <b>610</b>, a haptics system <b>620</b>, a sound system <b>630</b>, a lighting system <b>640</b>, a mechanical sensor <b>650</b>, a sound sensor <b>660</b>, and a light sensor <b>670</b>. The system <b>600</b> may also include a gyroscope <b>680</b>. The system <b>600</b> may be incorporated into an electronic device such as a cell phone, PDA, gaming device, etc.

0047The UI controller <b>610</b> may control the haptic system <b>620</b>, which may include a haptics driver and haptics actuator, to generate a haptic effect according to mechanical sensor <b>650</b> measurements as described above.

0048The UI controller <b>610</b> may also control a sound system <b>630</b> to generate an auditory effect. The sound system <b>630</b> may include an audio driver and a speaker. The auditory effect may be captured by the sound sensor <b>660</b>, for example a microphone. As described above with respect to haptic effects, the UI controller <b>610</b> may adjust the auditory effect based on the sound sensor <b>660</b> measurements.

0049Furthermore, the UI controller <b>610</b> may control a lighting system <b>640</b> to generate a visual effect, for example a change in the backlight of the display or touchscreen. The lighting system <b>640</b> may include display driver such as a LED/LCD driver. The visual effect may be captured by the light sensor <b>670</b>. As described above with respect to haptic effects, the UI controller <b>610</b> may adjust the visual effect based on the light sensor <b>670</b> measurements.

0050In an embodiment of the present invention, orientation information from the gyroscope <b>680</b> may be taken into account when generating sensory effects. For example, the gyroscope <b>680</b> may provide information regarding orientation state of the device. In turn, the UI controller <b>610</b> may generate or adjust a haptic command based on the orientation information.

0051Those skilled in the art may appreciate from the foregoing description that the present invention may be implemented in a variety of forms, and that the various embodiments may be implemented alone or in combination. Therefore, while the embodiments of the present invention have been described in connection with particular examples thereof, the true scope of the embodiments and/or methods of the present invention should not be so limited since other modifications will become apparent to the skilled practitioner upon a study of the drawings, specification, and following claims.

0052Various embodiments may be implemented using hardware elements, software elements, or a combination of both. Examples of hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so forth), integrated circuits, application specific integrated circuits (ASIC), programmable logic devices (PLD), digital signal processors (DSP), field programmable gate array (FPGA), logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth. Examples of software may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (API), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether an embodiment is implemented using hardware elements and/or software elements may vary in accordance with any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints.

0053Some embodiments may be implemented, for example, using a computer-readable medium or article which may store an instruction or a set of instructions that, if executed by a machine, may cause the machine to perform a method and/or operations in accordance with the embodiments. Such a machine may include, for example, any suitable processing platform, computing platform, computing device, processing device, computing system, processing system, computer, processor, or the like, and may be implemented using any suitable combination of hardware and/or software. The computer-readable medium or article may include, for example, any suitable type of memory unit, memory device, memory article, memory medium, storage device, storage article, storage medium and/or storage unit, for example, memory, removable or non-removable media, erasable or non-erasable media, writeable or re-writeable media, digital or analog media, hard disk, floppy disk, Compact Disc Read Only Memory (CD-ROM), Compact Disc Recordable (CD-R), Compact Disc Rewriteable (CD-RW), optical disk, magnetic media, magneto-optical media, removable memory cards or disks, various types of Digital Versatile Disc (DVD), a tape, a cassette, or the like. The instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, and the like, implemented using any suitable high-level, low-level, object-oriented, visual, compiled and/or interpreted programming language.