Free space pointing devices with tilt compensation and improved usability

ES 2 384 572 T3

DESCRIPTION

Pointing devices in free space with tilt compensation and improved usability

Background of the invention

[0001] The present invention relates generally to handheld pointing devices, and more specifically to free space pointing devices and the tilt compensation and improved usability techniques associated therewith.

[0002] The technologies associated with the communication of information have evolved rapidly over the last several decades. Television, mobile telephony, the Internet, and optical communication techniques (to mention just a few things) combine to inundate consumers with readily available information and entertainment options. Taking television as an example, the last three decades have seen the introduction of cable television service, satellite television service, pay-per-view movies, and video on demand. While viewers in the 1960s could typically receive perhaps four or five over-the-air TV channels on their television sets, today's viewers have the opportunity to choose from hundreds, thousands, and potentially millions of entertainment channels. and information. Video-on-demand technology, currently used primarily in hotels and similar venues, provides the potential for selecting home entertainment from thousands of movie titles.

[0003] The technological ability to deliver so much information and so much content to end users provides opportunities as well as challenges for system designers and service providers. One challenge is that while end users typically prefer to have more selection possibilities than less, this preference is outweighed by their desire for the selection process to be both quick and easy. Unfortunately, the development of the systems and interfaces through which end users access media articles has resulted in selection processes that are neither quick nor straightforward. Consider again the example of television shows. When television was in its infancy, determining which program to watch was a relatively straightforward process due primarily to the small number of options. A printed guide was consulted that was formatted, for example, as a series of columns and lines that showed the correspondence between (1) nearby television channels, (2) the programs that were broadcast on those channels, and (3) the date and the time. The television was tuned to the desired channel by adjusting a tuner button, and the viewer watched the selected program. Later, remote control devices were introduced that allowed viewers to tune in to the television from a distance. This addition to the user-television interface created the phenomenon known as “zapping” whereby a viewer could quickly view short segments broadcast on a number of channels to quickly find out which programs were available at any given time.

[0004] Despite the fact that the number of channels and the amount of viewable content have increased dramatically, over the last 30 years the structures and options of control devices that are in use as user interfaces have not changed much. Generally available for televisions. Printed guides are still the most predominant mechanism for transmitting programming information. The remote control with a plurality of up and down arrow buttons is still the most prevalent channel and content selection mechanism. The reaction of those who design and implement the user-TV interface to the increase in available media content has been a direct extension of existing selection procedures and infertility objects. Thus, the number of lines in the printed guides has been increased to accommodate more channels. The number of buttons on remote control devices has been increased to support additional functionality and content manipulation, eg. ex. as shown in Figure 1. However, this approach has significantly increased both the time required for a viewer to examine the available information and the complexity of the actions required to carry out a selection. Arguably, the cumbersome nature of the existing interface has hampered the commercial implementation of some services, such as. ex. video on demand, as consumers resist new services that add complexity to an interface that they already consider to be too slow and complex.

[0005] In addition to increases in bandwidth and content, the problem that constitutes the bottleneck of the user interface is being exacerbated by the aggregation of technologies. Consumers are reacting positively to the possibility of having the option to buy embedded systems instead of a series of segregated components. An example of this trend is the television / VCR / DVD combination in which three previously independent components are often sold today as an integrated unit. This trend is likely to continue, potentially with the end result that those of most if not all communication devices currently found in the home environment will be brought together as an integrated unit, eg. ex. a television / VCR / DVD / Internet access / radio / stereo unit. Even those who continue to buy standalone components will in all likelihood want seamless control of standalone components and interaction between them. With this increased aggregation comes the potential

ES 2 384 572 T3 for more complexity in the user interface. For example, when so-called remote (universal) units were introduced, p. ex. To combine the functionality of TV remote units and VCR remote units, the number of buttons on these universal remote units was typically greater than the number of buttons on the TV remote unit or the VCR remote unit individually. This added a number of buttons, and the functionality makes it very difficult to control anything other than the simplest aspects of a TV or VCR (VCR = VCR) without having to search for the exactly correct button on the screen. remote control. Many times, these universal remotes do not have enough buttons to access many levels of control or unique features for certain televisions. In these cases the original device remote unit is still required, and the original hassle of handling a plurality of remote controls remains due to user interface problems that stem from aggregation complexity. Some remote units have addressed this problem by adding “soft” buttons (“soft” buttons = programmable function buttons) that can be programmed with expert commands. These programmable function buttons sometimes have accompanying LCD displays to indicate their action. These also have the defect that they are difficult to use without looking away from the TV to look at the remote control. Yet another shortcoming of these remote units is the use of modes in an attempt to reduce the number of buttons. On these “mode” universal remote units there is a special button to select whether the remote control should communicate with the TV, DVD player, cable box, VCR, etc. This causes many usability problems including sending commands to the wrong device and requiring the user to look at the remote control to make sure it is in the correct mode, and does not simplify the integration of a plurality of devices. of devices. The more advanced of these universal remote units provide some integration by allowing the user to program command sequences for a plurality of devices into the remote control. This is such a difficult task that many users hire professional installers to program their universal remote units.

[0006] Some attempts have also been made to modernize the display interface between end users and media systems. However, these attempts typically suffer from, among other drawbacks, an inability to easily scale between large collections of media articles and small collections of media articles. For example, list-based interfaces may work well for small collections of media articles, but are heavy to browse for large collections of media articles. Interfaces that are based on hierarchical navigation (such as eg. tree structures) may be faster to navigate than list interfaces for large collections of media articles, but they are not easily adaptable to small collections of media articles. Additionally, users tend to lose interest in selection processes in which the user has to move through three or more layers in a tree structure. For all these cases, current remote units make this selection process even more cumbersome by forcing the user to repeatedly press the up and down buttons to navigate through the hierarchy list. When selection skip commands such as page up and page down are available, the user usually has to look at the remote control to find these special buttons, or has to learn to learn that they even exist. Consequently, in US Patent Application which has Deposit No. 10 / 768,432, was presented on January 30, 2004 and is titled "Control Structure with a Zoomable Graphical User Interface to Organize, Select and Launch Media Articles" organizational structures have been proposed, techniques and systems to simplify the command and display interface between users and media systems, as well as to speed up the selection process, while allowing service providers to take advantage of the increased bandwidth available to end-user equipment by facilitating the provision of a large number of media articles and new services to the user.

[0007] Of particular interest to this specification are remote devices usable to interact with such structures, as well as other applications and systems. As mentioned in the aforementioned application, several different types of remote devices can be used with such structures including, for example, guide balls, "mouse" pointing devices, glow sticks, and the like. However, another category of remote devices that can be used with such frameworks (and other applications) is free-space pointing devices. The phrase "pointer in free space" is used in this specification to refer to the ability of an input device to move in three (or more) dimensions in the air in front of a display screen, eg. For example, and to the corresponding user interface ability to transform these movements directly into user interface commands, such as. ex. movement of a cursor on the display screen. Data transfer between the free space pointing device and another device can be accomplished wirelessly or via a cable connecting the free space pointing device to the other device. Thus, the technique of "pointing in free space" devices differs from p. ex. pointing techniques with the conventional computer mouse that use a surface, such as eg. ex. a surface of a worktable or a mouse mat, as a surrogate surface from which the relative movement of the mouse is transformed into movement of the cursor on the computer display screen. In US Patent No. 5,440,326 an example of a pointing device can be found in free space.

[0008] The '326 patent describes, among other things, a vertical gyroscope which is adapted to be used as a pointing device to control the position of a cursor in the display unit of a computer. A

ES 2 384 572 T3 motor in the gyroscope core is suspended by two pairs of orthogonal gimbal suspensions of a hand controller device and is nominally oriented with its vertical axis of rotation by a pendulum device. Electro-optical shaft angle encoders detect the orientation of a handheld controller device as it is manipulated by a user, and the resulting electrical output is converted into a computer-usable format to control cursor movement on the unit's screen. visual representation of the computer.

[0009] However, the freedom of use associated with pointers in free space creates additional challenges. For example, since there is generally no substitute surface on which the pointing device rests in free space, the orientation of the hand control device can vary considerably from user to user or even from use to use. If a pointing device in free space is used to for example control the movement of a cursor displayed on a screen, then some transformation takes place between the detected movement of the handheld device and the movement of the cursor on the screen.

[0010] One technique to carry out this transformation is to use the frame of the body of the device as the frame of reference to transform the detected movement of the pointing device in free space into the intended movement of the cursor. The expression "body frame" refers to a set of axes associated with the body of the moving object as described in more detail below. Using the body's frame of reference to carry out the transformation, however, has certain drawbacks. For example, it requires the user to hold the device in a certain orientation in order to obtain the cursor movement that the user desires. For example, if the user holds the device sideways and moves the device from left to right, the cursor will move vertically, not horizontally, on the screen.

US 5,598,187 discloses a handheld pointing device having first and second motion detectors to detect horizontal and vertical movements of the device, which are transformed into movements of a cursor on a screen. The device further includes a rotation sensor to detect unwanted rotations when the user unconsciously twists his wrist while moving the device. The detected linear movement is corrected based on this detected rotation.

Summary

[0011] The systems and methods according to the present invention include pointing devices in free space that enhance usability by transforming motion data detected from a reference frame of the body associated with the pointing device in free space into motion data associated with a frame. user reference, wherein the effects associated with an inclination orientation relative to gravity in which the pointing device in free space is held by the user are eliminated.

[0012] Exemplary embodiments of the invention are defined in the attached independent claims 1 for a handheld pointing device and 4 for a method for operating a pointing device in free space. Advantageous embodiments of the invention are defined in the attached dependent claims 2, 3, 5 and 6.

Brief description of the drawings

The accompanying drawings illustrate embodiments of the present invention, where:

[0014] FIG. 1 represents a conventional remote control for an entertainment system;

[0015] FIG. 2 represents an example of a media system in which exemplary embodiments of the present invention can be implemented;

[0016] FIG. 3 shows a pointing device in free space according to an embodiment of the present invention;

[0017] FIG. 4 illustrates a view with a part removed from the pointing device in the free space of FIG. 3 including two rotation sensors and an accelerometer;

[0018] FIG. 5 is a block diagram illustrating data processing associated with pointing devices in free space according to an exemplary embodiment of the present invention;

[0019] FIGS. 6 (a) - 6 (d) illustrate the effects of tilting;

[0020] FIG. 7 represents a hardware architecture (hardware = physical support) of a pointing device in free space according to an embodiment of the present invention;

[0021] FIG. 8 is a state diagram representing a stationarity detection mechanism according to an embodiment of the present invention;

ES 2 384 572 T3

[0022] FIG. 9 is a block diagram illustrating the transformation of detected motion data from a first frame of reference to a second frame of reference according to an exemplary embodiment of the present invention; Y

[0023] FIG. 10 graphically illustrates the transformation of detected motion data from a first frame of reference to a second frame of reference according to an embodiment of the present invention.

Detailed description

The following detailed description of the invention refers to the accompanying drawings. Like reference numbers in different drawings identify elements or the like. Also, the following detailed description does not limit the invention. Rather, the scope of the invention is as defined by the appended claims.

In order to establish context for this discussion, an example aggregate media system 200 in which the present invention may be implemented will first be described with respect to Figure 2. Those skilled in the art will understand, however, that the present invention is not limited to implementation in this type of media system, and that more or fewer components may be included therein. There, an input / output (I / O) bus 210 connects the system components together in the media system 200. The I / O bus 210 represents any of a number of different mechanisms and techniques for routing signals between components of the media system. For example, I / O bus 210 may include an appropriate number of separate audio "patch" cables that route audio signals, coaxial cables that route video signals, two-wire serial lines, or infrared or radio frequency transceivers that route signals. control, fiber optics or any other routing mechanisms that route other types of signals.

In this exemplary embodiment, the media system 200 includes a television / monitor 212, a video cassette recorder (VCR) 214, a digital video disc (DVD) recorder / player 216, an audio / audio tuner. video 218 and a compact disc player 220 coupled to I / O bus 210. The VCR 214, the dVd recorder / player 216, and the compact disc player 220 may be single disc or single cassette devices, or alternatively they may be multiple disc or multiple cassette devices. These units can be independent units, or they can be integrated into a set. In addition, the media system 200 includes a microphone and speaker system 222, a video camera 224, and a wireless I / O control device 226. According to exemplary embodiments of the present invention, the wireless I / O control device 226 is a pointing device in free space according to one of the exemplary embodiments described below. Wireless I / O control device 226 can communicate with entertainment system 200 using, eg. For example, an infrared or radio frequency transmitter or transceiver. Alternatively, the I / O control device may be connected to the entertainment system 200 via a cable.

The entertainment system 200 also includes a system controller 228. In accordance with an exemplary embodiment of the present invention, the system controller 228 serves to store and display available entertainment system data from a plurality of sources. entertainment system data and to control a wide variety of features associated with each of the system components. As shown in Figure 2, the system controller 228 is directly or indirectly coupled to each of the system components, as needed, through the I / O bus 210. In addition to the I / O bus 210 or Rather, the system controller 228 is configured with a wireless communication transmitter (or transceiver) that is capable of communicating with system components via infrared signals or radio frequency signals. Regardless of the control means, the system controller 228 is configured to control the media components of the media system 200 via a graphical user interface described below.

As further illustrated in Figure 2, the media system 200 may be configured to receive media articles from various media sources and service providers. In this exemplary embodiment, the media system 200 receives media input from and optionally sends information to any or all of the following sources: cablecast 230, satellitecast 232 (eg. via satellite antenna), very high frequency (VHF) or ultra-high frequency (UHF) radio frequency communication of 234 television broadcast networks (e.g. via aerial antenna), network phone 236 and cable modem 238 (or other Internet content source). Those skilled in the art will understand that the media components and media sources illustrated and described with respect to Figure 2 are purely exemplary and that the media system 200 may include a greater or lesser number of both. For example, other types of inputs to the system include AM / FM radio and satellite radio.

[0029] More details regarding this example of an entertainment system and associated structures can be found in the aforementioned US Patent Application entitled "Control structure with a zoomable graphical user interface for organizing, selecting and launching articles from the media". As an alternative,

ES 2 384 572 T3 remote devices according to the present invention can be used in conjunction with other systems, such as computer systems including, e.g. eg, a display unit, a processor and a memory system, or with various other systems and applications.

[0030] As mentioned in the Background portion, remote devices that operate as pointers in free space are of particular interest to the present specification. Such devices allow the transformation of movement, such as eg. ex. gestures, in commands to a user interface. An example of a pointing device in free space 400 is represented in Figure 3. There, the movement of the pointing device in free space produced by the user can be defined, for example, in terms of a combination of posture movement about the x-axis (roll), lift about the y-axis (tilt) and / or orientation with respect to the z axis (yaw) of the pointing device in free space 400. In addition, some embodiments of the present invention may also measure the linear movement of the pointing device in free space 400 along the x, y, and z axes to generate cursor movement commands or other user interface commands. In the embodiment of Figure 3, the free space pointing device 400 includes two buttons 402 and 404 as well as a scroll wheel 406, although other embodiments will include other physical configurations. According to embodiments of the present invention, it is envisaged that the pointing devices in free space 400 will be held by a user in front of a display unit 408 and that the movement of the pointing device in free space 400 will be transformed by the device. pointer in free space in an output that is usable to interact with the information displayed on the display unit 408, e.g. ex. to move the cursor 410 on the display unit 408. For example, the rotation of the pointing device in free space 400 about the y-axis can be detected by the pointing device in free space 400 and transformed into an output capable of be used by the system to move cursor 410 along the y2 axis of display unit 408. Similarly, the rotation of the pointing device in the free space 408 about the z-axis can be detected by the pointing device in the free space 400 and transformed into an output that can be used by the system to move the cursor 410 along the x2 axis of display unit 408. It will be understood that the output of the pointing device in free space 400 can be used to interact with the display unit 408 in a number of ways other than (or in addition to) cursor movement, and thus for example said output can control the movement of the cursor. cursor fading, volume, or media transport (play, pause, fast forward, and rewind). Entry orders can include other operations in addition to the movement of the cursor, such as an increase or decrease in a certain area of ​​a visualization. A cursor may or may not be visible. Similarly, the rotation of the pointing device in free space 400 detected about the x-axis of the pointing device in free space 400 can be used in addition to or as an alternative to rotation around the y-axis and / or the z-axis to provide it with a input to a user interface.

[0031] According to an embodiment of the present invention, as shown in Figure 4, two rotation sensors 502 and 504 and an accelerometer 506 can be used as sensors in the pointing device in free space 400. The rotation sensors 502 and 504 can for example be implemented using ADXRS150 or ADXRS401 sensors manufactured by Analog Devices. Those skilled in the art will understand that other types of rotation sensors may be used as rotation sensors 502 and 504, and that sensors ADXRS150 and ADXRS401 are used purely by way of illustrative example. Unlike traditional gyroscopes, ADXRS150 rotation sensors use MEMS technology (MEMS = micro-electromechanical systems) to have a resonant mass that is attached to a structure in such a way that it can resonate in only one direction. The resonant mass is displaced when the body to which the sensor is attached is rotated about the sensor axis of the sensor. This displacement can be measured using the Coriolis acceleration effect to determine an angular velocity associated with a rotation along the sensing axis. If the rotation sensors 502 and 504 have a single sensor axis (such as the ADXRS150s), then they can be mounted on the pointing device in free space 400 such that their sensor axes are aligned with the rotations to be measured. For this exemplary embodiment of the present invention, this means that the rotation sensor 504 is mounted such that its sensor axis is parallel to the y-axis, and that the rotation sensor 502 is mounted such that its sensor axis is parallel to the z axis, as shown in Figure 4. Note, however, that it is not required to align the sensor axes of the rotation sensors 502 and 504 parallel to the desired measurement axes, since embodiments of the present invention also provide techniques to compensate for inter-axis deviation.

[0032] A challenge in implementing examples of pointing devices in free space 400 according to the present invention is to employ components, such as eg. ex. rotation sensors 502 and 504, which are not too expensive, while at the same time providing a high degree of correlation between the movement of the pointing device in free space 400, the user's expectations regarding how the user interface will react to that particular movement of the pointing device in free space, and the actual performance of the user interface in response to that movement. For example, if the pointing device in free space 400 is not moving, the user is most likely hoping that the cursor does not advance across the screen. Similarly, if the user rotates the pointing device in free space 400 purely about the y-axis, he will presumably not expect to see that the resulting cursor movement in the display unit 408 contains any significant x2-axis components. To achieve these and other aspects of exemplary embodiments of the present invention,

ES 2 384 572 T3 are carried out by the handheld device 400 various measurements and various calculations that are used to adjust the outputs of one or more of the sensors 502, 504 and 506 and / or as part of the input used by a processor to determine an appropriate output for the user interface based on the outputs of the sensors, 502, 504, and 506. These measurements and calculations are used to compensate for factors that broadly fall into two categories: (1) factors that are intrinsic to the pointing device in free space 400, such as eg. ex. the errors associated with the particular sensors 502, 504, and 506 used in device 400, or the way the sensors are mounted in device 400 and (2) factors that are not intrinsic to the pointing device in free space 400 but are instead associated with how the user is using the pointing device in free space 400, such as eg. ex. linear acceleration, tilt, and tremor. Examples of techniques to handle each of these effects are described below.

Illustrated in Figure 5 is a process model 600 that describes the general operation of pointing devices in free space according to embodiments of the present invention. Rotation sensors 502 and 504, as well as accelerometer 506, produce analog signals that are periodically sampled, e.g. ex. at a rate of 200 samples / second. For the purposes of this discussion, a set of these inputs will be referred to using the notation (x, y, z, αγ, az), where x, y, z are the sampled output values ​​from the 506 triaxial accelerometer example that are associated with the acceleration of the pointing device in free space in the directions of the x-axis, the y-axis, and the z-axis, respectively, ay is the sampled output value of the rotation sensor 502 associated with the rotation of the pointing device in free space around the y-axis, and az is the sampled output value of the rotation sensor 504 associated with the rotation of the pointing device in free space 400 about the z axis.

[0034] The output of the accelerometer 506 is given and, if the accelerometer 506 gives an analog output, then the output is sampled and digitized by an A / D converter (not illustrated) to generate the sampled accelerometer output 602. The values ​​of Sampled outputs are converted to be passed from raw units to acceleration units, such as p. ex. Gravities (g), as indicated by the conversion function 604. The acceleration calibration block 606 provides the values ​​that are used for the conversion function 604. This calibration of the accelerometer output 602 may include, for example, an offset of one or more of the group members consisting of errors scale, offset, and axis misalignment associated with the 506 accelerometer. Examples of accelerometer data conversions can be performed using the following equation:

A = S * ((MP). * G (T)) (1) where M is a 3x1 column vector that is composed of the sampled output values ​​(x, y, z), P is a column vector of 3x1 of sensor offsets, and S is a 3x3 matrix containing compensation for both scale and shaft misalignment and sensor rotation. G (T) is a gain factor that is a function of temperature. The “*” operator represents matrix multiplication and the “. *” Operator represents element multiplication. The accelerometer example 506 has a full range example of +/- 2g. Sensor offset, P, refers to sensor output, M, for an accelerometer measurement of 0g. "Scale" refers to the conversion factor between the value in the sampling units and g. The true scale of any given accelerometer sensor may deviate from these nominal scale values ​​due to, e.g. eg, to manufacturing variances. Consequently, the scale factor in the above equations will be proportional to this deviation.

The scale and offset deviations of the accelerometer 506 can be measured, for example, by applying 1 g of force along an axis and measuring the result, R1. Then a force of -1 g is applied, obtaining the measurement R2 as a result. The individual axis scale, s, and the individual axis deviation, p, can be computed as follows:

s = (R1 - R2) / 2 (2) p = (R1 + R2) / 2 (3)

In this simple case, P is the column vector of p for each axis, and S is the 1 / s diagonal matrix for each axis.

However, in addition to scale and offset, the readings generated by accelerometer 506 can also suffer from cross-axis effects. Cross axis effects include non-aligned axes, eg. ex. wherein one or more of the sensing axes of the accelerometer 506 as it is (are) mounted on the pointing device in free space 400 are not aligned with the corresponding axis in the inertial frame of reference, or to mechanical errors associated with the mechanization of the accelerometer itself 506, p. ex. where even though the axes are correctly aligned, a pure y-axis acceleration force can result in a sensor reading along the z-axis of the 506 accelerometer. These two effects can also be measured and added to the calibration which is carried out by function 606.

[0037] The accelerometer 506 serves various purposes in examples of pointing devices in free space according to embodiments of the present invention. For example, if rotation sensors 502 and 504 are implemented using the examples of Coriolis rotation sensors described above, then the output of the

ES 2 384 572 T3 rotation sensors 502 and 504 will vary based on the linear acceleration experienced by each rotation sensor. Thus, an example of the use of the accelerometer 506 is to compensate for fluctuations in the readings generated by the rotation sensors 502 and 504 that are caused by variances of linear acceleration. This can be accomplished by multiplying the converted accelerometer readings by a 610 gain matrix and subtracting (or adding) the results from (or to) the corresponding 612 sampled rotation sensor data. For example, the sampled rotation data to and from the sensor rotation speed 502 can be compensated for linear acceleration in block 614 as follows:

ay '= ay - C * A (4) where C is the 1x3 row vector of the rotational sensor's susceptibility to linear acceleration along each axis given in units / g and A is the calibrated linear acceleration. Additionally, linear acceleration compensation for the az-sampled rotation data from the rotation sensor 504 can be provided in block 614. The gain matrices, C, vary between rotation sensors due to manufacturing differences. C can be computed using the mean value for many rotation sensors, or it can be computed individually for each rotation sensor.

[0038] Like the accelerometer data, the sampled rotation data 612 is then converted so as to be passed from a value in sampling units to a value associated with an angular rotation speed, such as p. ex. radians / sec, in function 616. This conversion step may also include a calibration provided by function 618 to compensate for rotational data sampled for eg. eg, scale and deviation. Conversion / calibration for both ay and az can be carried out using, for example, the following equation:

α rad / sec. = (a '- deviation (T)) * scale + dDeviation (5) where a' refers to the value that is converted / calibrated, deviation (T) refers to a deviation value associated with temperature, scale refers to the conversion factor between the value in sampling units and rad / sec, and deviation refers to a dynamic deviation value. Equation (5) can be implemented as a matrix equation, in which case all variables are vectors except scale. In matrix equation form, the scale corrects for shaft misalignment and rotational deviation factors. Each of these variables is discussed in more detail below.

[0039] The deviation deviation (T) and dDeviation values ​​can be determined in a number of different ways. When the free space pointing device 400 is not being rotated in, for example, the y-axis direction, the sensor 502 should output its offset value. However, drift can be highly affected by temperature, so this drift value will likely vary. Calibration of offset for temperature can be performed at the factory, in which case the value (s) for offset (T) can be pre-programmed into handheld 400, or As an alternative, the offset calibration for temperature can also be learned dynamically over the lifetime of the device. To carry out dynamic deviation compensation, an input from a temperature sensor 619 is used in the rotation calibration function 619 to compute the current value for deviation (T). The deviation parameter (T) removes most of the deviation bias from sensor readings. However, negating almost all cursor advancement for zero movement can be useful in producing a high performance pointing device. Accordingly, the additional factor dDeviation can be dynamically computed while the pointing device is in use in free space 400. The stationarity detection function 608 determines when the handheld device is most likely stationary and when the deviation should be recalculated. Examples of techniques for implementing stationarity detection function 608, as well as other uses for it, are described below.

[0040] An example dDeviation computing implementation employs calibrated sensor outputs that are filtered on a low pass filter. The stationarity output detection function 608 provides the rotation calibration function 618 with an indication to start computing, for example, the mean of the low-pass filter output. The stationarity output detection function 608 may also control when the newly computed mean is factored into the existing value for dDeviation. Those skilled in the art will recognize that a multitude of different techniques can be used to compute the new value for dDeviation from the existing value of dDeviation and the new mean, including but not limited to simple averaging, low-pass filtration, and Kalman filtration. Additionally, those skilled in the art will recognize that numerous variations may be employed for offset compensation of the rotation sensors 502 and 504. For example, the offset function (T) may have a constant value (eg, invariant with the temperature), more than two offset offset values ​​may be used and / or only a single offset value may be computed / used for offset offset.

[0041] After conversion / calibration in block 616, the inputs from the rotation sensors 502 and 504 can be further processed to rotate those inputs to an inertial frame of reference, that is, to compensate for the tilt associated with the way how the user is holding the pointing device in space

ES 2 384 572 T3 free 400, in function 620. The correction of the inclination is another important aspect of some embodiments of the present invention inasmuch as it is intended to compensate for the differences in the use patterns of the pointing devices in free space according to the present invention. More specifically, the tilt correction according to exemplary embodiments of the present invention is intended to compensate for the fact that the users will hold the pointing devices in different rotational positions along the x-axis, but that the sensor axes of the Rotation sensors 502 and 504 in the free space pointing devices 400 are fixed. It is desirable that the translation of the cursor through the display unit 408 is considerably insensitive to how the user holds the pointing device in the free space 400, and so on. For example, rotation in both directions of the pointing device in the free space 400 in a way that corresponds generally to the horizontal dimension (axis X2) of the display unit 408 should result in a translation of the cursor along the axis x2, while rotating the pointing device in free space up and down in a manner that generally corresponds to the vertical dimension (y2 axis) of the display unit 408 should result in a translation of the cursor along the axis y2, regardless of the orientation in which the user is holding the pointing device in free space 400.

[0042] To better understand the need for tilt compensation according to embodiments of the present invention, consider the example shown in Figure 6 (a). There, the user is holding the pointing device in free space 400 in an example of an inertial frame of reference that can be defined as one having an x-axis rotation value of 0 degrees, and so on. For example, the inertial frame of reference can be one in which the pointing device in free space has its bottom practically parallel to a floor of a room in which p is located. ex. a TV. The inertial frame of reference may, purely as an example, correspond to the orientation that is illustrated in Figure 6 (a), or it may be defined as any other orientation. The rotation of the pointing device in the free space 400 in the directions of the y-axis of the z-axis will be detected by the rotation sensors 502 and 504, respectively. For example, rotating the pointing device in free space 400 about the z axis by an amount Δζ as shown in Figure 6 (b) will result in a corresponding translation of the cursor ΔΧ2 in the dimension of the X2 axis through the graphing unit 408 (ie, the distance between the dashed version of the cursor 410 and the non-dashed version).

If, on the other hand, the user holds the pointing device in the free space 400 in a different orientation, eg. ex. with a certain amount of rotation along the x-axis with respect to the inertial frame of reference, then the information provided by sensors 502 and 504 would not provide (in the absence of tilt compensation) an accurate representation of the interface actions pursued by the user. For example, referring to Figure 6 (c), consider a situation in which the user holds the pointing device in free space 400 with an x-axis rotation of 45 degrees with respect to the example inertial frame of reference that illustrated in Figure 6 (a). Assuming that the user is imparted by the user to the pointing device in free space 400 the same rotation along the z axis Δζ as in the example of Figure 6 (b), the cursor 4l0 will instead be translated both in the direction of the x2 axis as in the direction of the y2 axis as shown in Figure 6 (d). This is due to the fact that the sensor axis of the rotation sensor 502 is now oriented between the y-axis and the z-axis (due to the orientation of the device in the user's hand). Similarly, the sensing axis of rotation sensor 504 is also oriented between the y-axis and the z-axis (albeit in a different quadrant). In order to provide an interface that is transparent to the user in terms of how the pointing device is held in free space 400, tilt compensation according to embodiments of the present invention transforms the output of the readings from the rotation sensors 502 and 504, bringing it back to the inertial frame of reference as part of the processing of the readings from these sensors for their transformation into information indicative of the rotational movement of the pointing device in free space 400.

[0044] According to exemplary embodiments of the present invention, returning to Figure 5, this can be accomplished by determining the inclination of the pointing device in free space 400 using inputs y and z received from accelerometer 506 in function 622. More specifically, after the acceleration data has been converted and calibrated as described above, it can be low-pass filtered on the LPF (LPF = low-pass filter) 624 to provide you with an acceleration (gravity) value. mean to the slope determination function 622. Then, the slope θ can be calculated in function 622 as:

¢ = tg-Q (7)

The value θ can be computed numerically as atg2 (y, z) to prevent division by zero and give the correct sign. Then function 620 can perform the rotation R of the converted / calibrated inputs a and z using the equation:

<img file="ES2384572T3_D0001.tif" />

ES 2 384 572 T3 to rotate the converted / calibrated inputs ay and az to compensate for the tilt θ. Tilt compensation as described in this exemplary embodiment is a subset of a more general technique for translating sensor readings from the body's frame of reference to a user's frame of reference according to another embodiment hereof. invention described below.

[0045] Once the calibrated sensor readings have been compensated for linear acceleration, transformed into readings indicative of the angular rotation of the pointing device in free space 400, and compensated for tilt, post-processing can be performed on the blocks. 626 and 628. Post-processing examples can include compensation for various factors such as human tremor. Although shaking can be eliminated using several different methods, one way to eliminate shaking is to use hysteresis. The angular velocity produced by the rotation function 620 is integrated to produce an angular position. Hysteresis of a calibrated quantity is then applied to the angular position. The derivative of the hysteresis block output is taken to again produce an angular velocity. The resulting output is then scaled in function 628 (p. ex. based on the sampling period) and used to generate a result within the interface, such as p. ex. the movement of a cursor 410 in a display unit 408.

[0046] Having given a description of the process of examples of pointing devices in free space according to the present invention, Figure 7 illustrates an example of hardware architecture. There, a processor 800 communicates with other elements of the pointing device in free space including a scroll wheel 802, a JTAG 804, LEDs 806, a matrix switcher 808, an IR photodetector 810, rotation sensors 812, an accelerometer. 814 and a 816 transceiver. The scroll wheel 802 is an optional input component that allows the user to input inputs to the interface by rotating the scroll wheel 802 clockwise or counterclockwise. The JTAG 804 provides the programming and debugging interface to the processor. LEDs 806 provide visual feedback to the user, for example when a button is pressed. The matrix switcher 808 receives inputs, such as eg. ex. indications that a pointing device button has been pressed or released in free space 400, said inputs then being passed to processor 800. The optional IR photodetector 810 may be provided to enable the exemplary pointing device in free space to learn IR codes from other remote controls. Rotation sensors 812 provide the processor with 800 relative readings eg. ex. to rotation along the y-axis and along the z-axis of the pointing device in free space as described above. The accelerometer 814 provides the processor 800 with readings relative to the linear acceleration of the pointing device in free space 400, which can be used as previously described, eg. ex. to perform tilt compensation and to compensate for errors that linear acceleration introduces in the rotation readings generated by the rotation sensors 812. The transceiver 816 is used to communicate information to and from the pointing device in free space 400 , p. ex. to the system controller 228 or to a processor associated with a computer. The 816 transceiver may be a wireless transceiver which e.g. ex. work according to Bluetooth standards for short-range wireless communication, or an infrared transceiver. Alternatively, the free space pointing device 400 can communicate with the systems via a wireline connection.

[0047] In the embodiment of Figure 4, the free space pointing device 400 includes two rotation sensors 502 and 504, as well as an accelerometer 506. However, according to another embodiment of the present invention, a pointing device in free space may alternatively include only one rotation sensor, e.g. ex. to measure angular velocity in the direction of the z-axis, and an accelerometer. For an exemplary embodiment of this type, a functionality similar to that described above can be provided by using the accelerometer to determine the angular velocity along the axis that is not detected by the rotation sensor. For example, the speed of rotation around the y-axis can be computed using data generated by the accelerometer and calculating:

<img file="ES2384572T3_D0002.tif" />

Furthermore, parasitic acceleration effects that are not measured by a rotation sensor must also be eliminated. These effects include actual linear acceleration, measured acceleration due to rotational speed and rotational acceleration, and acceleration due to human tremor.

The stationarity detection function 608, which has been briefly mentioned above, can operate to determine whether the pointing device in the free space 400 is for example stationary or activated (in motion). This categorization can be carried out in a number of different ways. One way, according to an exemplary embodiment of the present invention, is to compute the variance of the input data sampled from all inputs (x, y, z, αy, oz) within a predetermined window, such as p. ex. every quarter of a second. This variance is then compared with a threshold to classify the pointing device in free space as stationary or as active.

ES 2 384 572 T3

[0049] Another technique for detecting stationarity according to embodiments of the present invention involves transforming the inputs in order to bring them to the frequency domain carrying p. ex. perform a Fast Fourier Transform (FFT) on the input data. Then the data can be analyzed using p. ex. peak detection methods to determine whether the free space pointing device 400 is in a steady state or an active state. Additionally, a third category can be distinguished, which is specifically the case in which the user is holding the pointing device in free space 400 but does not move it (this state is also called "stable" state here). This third category can be distinguished from the stationary (unsupported) and active states by detecting the small movement of the pointing device in free space 400 that is introduced by the shaking of the user's hand when the pointing device in free space 400 is being held. by a user. Peak detection can also be used by stationarity detection function 608 to make this determination. The peaks that are located within the human tremor frequency range, which is p. ex. nominally 8-12 Hz, they will typically exceed the device noise floor (experienced when the device is stationary and not being sustained) by approximately 20 dB.

In the previous examples, the variances in the frequency domain were detected within a particular frequency range, although the actual frequency range to be monitored and used to characterize the state of the pointing device in free space 400 may vary. For example, the nominal tremor frequency range may vary based on p. ex. ergonomics and weight of pointing device in free space 400, going p. ex. 8-12 Hz to 4-7 Hz.

According to another embodiment of the present invention, the stationarity detection mechanism 608 may include a state machine. An example of a state machine is shown in Figure 8. There, the ACTIVE state is in this example the default state during which the pointing device in free space 400 is in motion and is being used for p. ex. supply inputs to a user interface. Pointing device in free space 400 may enter the ACTIVE state upon device power connection as indicated by the reset input. If the pointing device in free space 400 stops moving, it may then enter the IDLE state. The various transitions between states illustrated in Figure 8 can be initiated by any of a number of different criteria including, but not limited to, a data output from one of the rotation sensors 502 and 504. or both, an accelerometer 506 data output, time domain data, frequency domain data, or any combination thereof. Transition conditions between states will be referred to generically herein using the convention "ConditionstateA .stateB". For example, the pointing device in free space 400 will go from the ACTIVE state to the INACTIVE state when the active.inactive condition occurs. For purely illustrative purposes, consider that in an example of a pointing device in free space 400 the active.inactive condition can occur when values ​​of the mean and / or standard deviation of both the rotation sensor (s) and the accelerometer falls below first predetermined threshold values ​​for a first predetermined period of time.

The transitions between states can be determined by a number of different conditions based on the interpreted sensor outputs. Examples of conditional metrics include the variance of the interpreted signals within a time window, the threshold value between a reference value and the interpreted signal within a time window, the threshold between a reference value and the filtered interpreted signal. within a time window and the threshold between a reference value and the interpreted signal from a start-up time, that can be used to determine transitions between states. All of these conditional metrics or any combination of them can be used to initiate transitions between states. As an alternative, other metrics can also be used. According to an embodiment of the present invention, a transition from the INACTIVE state to the ACTIVE state occurs when (1) an average value of a sensor output or sensor outputs within a time window is greater than a predetermined threshold or that predetermined thresholds, or (2) a variance of sensor output values ​​(s) within a time window is greater than the predetermined threshold or than the predetermined thresholds, or (3) an instantaneous delta between sensor values ​​is greater than a predetermined threshold.

The INACTIVE state allows the stationarity detection mechanism 608 to distinguish between brief pauses during which the free space pointing device 400 is still in use, being the same p. ex. on the order of a tenth of a second, and a real transition to a stable or stationary condition. This protects against the functions that are carried out during the STABLE and STABLE states, described below, from being carried out inadvertently when the pointing device is being used in free space. The pointing device in free space 400 will return to the ACTIVE state when the condition is inactive. ex. if the pointing device in the free space 400 begins to move again such that the measured outputs of the rotation sensor (s) and the accelerometer exceed the first threshold before a predetermined period of time has elapsed in the INACTIVE status.

The free space pointing device 400 will enter the STABLE state or the STATIONARY state after the second predetermined period of time has elapsed. As mentioned above, the STABLE state reflects the characterization of the pointing device in free space 400 as that of a device that is

ES 2 384 572 T3 being held by a person but practically not in motion, while the STATIONARY state reflects a characterization of the pointing device in free space as that of a device that is not being held by a person. Thus, an example of a state machine according to the present invention can lead to a transition to the STABLE state after the second predetermined period of time has elapsed if minimal movement associated with a hand shake is present, or else it can give place to go to the STATIONARY state.

[0055] The STABLE and STATIONARY states define periods of time during which the pointing device in free space 400 can carry out various functions. For example, since the STABLE state is intended to reflect the time periods in which the user is holding the pointing device in free space 400 but is not moving it, the device can record the movement of the pointing device in free space 400 when it is in the STABLE state p. ex. storing outputs from the rotation sensor (s) and / or the accelerometer while the device is in this state. These stored measurements can be used to determine a tremor pattern associated with a specific user or specific users as described below. Similarly, when in the STATIONARY state, the free space pointing device 400 can take readings from the rotation sensors and / or the accelerometer to use to compensate for drift as described above.

[0056] If the pointing device in free space 400 begins to move while in the STABLE or STABLE state, this may initiate a return to the ACTIVE state. Otherwise, after measurements have been taken, the device may go into the SLEEP state. While in the sleeping state, the device can enter a power consumption reduction mode in which the power consumption of the pointing device in free space is reduced and p. ex. the sampling rate of the rotation sensors and / or the accelerometer is also reduced. The SLEEP state can also be entered by means of an external command, whereby the user or other device can command the pointing device in free space 400 to enter the SLEEP state.

[0057] Upon receiving another command, or if the pointing device in free space 400 begins to move, the device may go from the SLEEP state to the AWAKE state. Like the INACTIVE state, the AWAKE state provides an opportunity for the device to confirm that a transition is warranted to go to the ACTIVE state, ie p. ex. that the pointing device in free space 400 was not inadvertently pushed.

The conditions for the transitions between states can be symmetric or they can be different. Thus, the threshold associated with the condition_toinactive-.inactive may be equal to (or different from) the threshold (s) associated with the inactive-.active condition. This enables free space pointing devices according to the present invention to more accurately capture user input. For example, exemplary embodiments that include a state machine implementation allow, among other things, that the threshold for entering a stationary condition is different from the threshold for exiting a stationary condition.

[0059] Entering a state or exiting a state can be used to initiate other functions of the device as well. For example, the user interface can be connected on the basis of a transition from any state to the ACTIVE state. Conversely, the free space pointing device and / or the user interface can be disconnected (or enter a sleep mode) when the free space pointing device goes from an ACTIVE or STABLE state to a STATIONARY or INACTIVE state. . Alternatively, the cursor 410 may be displayed or removed from the screen based on the transition out of steady state or the transition to steady state of the pointing device in free space 400.

As mentioned above, exemplary embodiments of the present invention process the movement data received from the sensor (s) of the pointing device in free space to translate this data from the reference frame of the body of the device. pointer in free space to another frame of reference, such as p. ex. the user's frame of reference. In the application example of a pointing device in free space in which said device is used to control a user interface displayed on a screen, such as p. ex. in a television, the user's frame of reference could be a coordinate system associated with the television screen. Regardless, moving the data from the body's frame of reference to another frame of reference improves the usability of the handheld device resulting in operation that is from the user's perspective rather than the device's perspective. Thus, when the user moves his hand from left to right in front of a display unit while holding the pointing device in free space, the cursor will move from left to right regardless of the orientation of the pointing device in free space.

[0061] To simplify this discussion, an example of a processing system associated with a pointing device in free space as described in greater detail above is shown in Figure 9. There, the handheld system detects movement using one or more 901 sensors, such as eg. ex. rotation sensor (s), gyroscope (s), accelerometer (s), magnetometer (s), optical sensor (s), camera (s), or any combination thereof. The

Sensors are then interpreted at block 902 to produce an assessment of the movement that occurred. Processing block 903 then transforms the measured movement from the device's natural (body) frame of reference to the user's frame of reference. The movement is then transformed at 904 into meaningful actions that are interpreted at block 905 and sent to the system to produce a meaningful response, such as moving a screen cursor.

[0062] Block 903 converts the detected movement into the user's frame of reference rather than the device's frame of reference. Orientation can be represented by many different mathematically similar methods including Euler angles, a directing cosine matrix (DCM), or a unit quaternion. Position is generally represented as a deviation from the origin of the coordinate system in consistent units that include, but are not limited to, the members of the group consisting of meters, centimeters, feet, inches, and miles. In an exemplary embodiment described above, a pointing device in free space measures inertial forces including acceleration and rotational speed. These forces are measured with respect to the body of the device by sensors mounted therein. In order to translate the measured data into the user's frame of reference, the device calculates both its position and its orientation.

In this exemplary embodiment, it is assumed that the user's frame of reference is stationary and has a fixed orientation, although those skilled in the art will understand that this technique according to the present invention can easily be extended to cases in which the user's frame of reference is non-stationary by performing a direct transformation to move to the frame of temporal variation or by passing first to a stationary frame and then moving to the mobile frame. For the example of the fixed and stationary orientation user frame of reference, the conversion to move from the body frame to the user frame can be accomplished using the following equations:

Pu = Rotate (Pb, Q) + Pdelta

Pu '= Rotate (Pb', Q)

Pu ”= Rotate (Pb '', Q) Wu = Rotate (Wb, Q) Wu '= Rotate (Wb', Q) where:

Rotate represents the quaternion rotation operator such that Rotate (A, Q) is equal to Q * AQ where Q * is the conjugate of the quaternion and vector A is a quaternion with the complex component equal to A and the real component equal to 0;

Pu is the position in the user's frame of reference;

Pb is the position in the frame of reference of the device;

'represents the derivative. Therefore, Pu 'is the derivative of the position in the user's frame of reference which is the velocity in the user's frame of reference;

Wu is the angular velocity of the device at body angles in the user's frame of reference; Wb is the angular velocity of the device at body angles in the frame of the device body;

Pdelta is the difference between the origin of the user's reference frame and the body's reference frame in the user's reference frame coordinate system;

Q is the normalized rotation quaternion representing the rotation of the body frame to the user's frame. Since the rotation quaternion to rotate from user frame to body frame is Q *, we could substitute R * for Q, where R is the rotation from user frame to body frame. Note that Q can be represented in a number of equivalent forms including Euler angles and the directing cosine matrix (DCM), and that the above equations can vary slightly in their equivalent forms based on different representations of Q. Figure 10 graphically illustrates the transformation from a body frame of reference to a user frame of reference.

[0064] During operation, the device calculates Q in an implementation-dependent manner to carry out this transformation. An example of implementation described above involves compensating for tilt (that is, variations in the x-axis roll of the pointing device in free space based on how it is held by a user). The orientation is computed by first calculating the component of acceleration due to gravity in the frame of the body, Ab. By definition, the vector of acceleration due to gravity in the user frame, Ag, is set to [0, 0, -1]. Since gravity cannot calculate orientation (rotation about the z-axis), the body frame assessment is used for orientation. Consequently, the quaternion of rotation has an axis of rotation in the plane z = 0. The following is one of several mathematically equivalent methods for computing the rotation quaternion:

V = || Ab || x || Ag || (cross product of unit vectors) qV = || v || a = sin '' | V |

ES 2 384 572 T3

Q = Fourth mion [qV, a] = [qV * sin (a / 2), cos (a / 2)]

The position is then computed as the double integral of the acceleration in the user's frame. The acceleration in the user frame is the acceleration of the body frame rotated to the user frame by the aforementioned Q. The origin is normally assumed to be zero when the device is first activated, but the origin can be reset manually or automatically during normal operation.

[0065] In general, when the device is not in motion, Pu ', Pu ", Wu and Wu" are all 0. In this embodiment, Pb' and Wb are measured. Although there is an infinite number of rotations Q, the minimum rotation can be selected from the available set and can be used to calculate Wu on the basis of Wb. Alternatively, Q can be computed using a presumed starting deviation orientation Qo, integrating Wb over time as shown using the discrete time integral given below:

Wb angle = | Wb) * period

Qdelta = Quatemionf Wb, Wbangle] = [|| Wb j | * sin (Wbangle / 2), cos (Wbangle / 2) J

Qnext ~ Qo ** Qdelta

Where * represents multiplication and ** represents multiplication of quaternions. Additional stability can be provided by constant field vectors including gravity and the earth's magnetic field, and can be combined with the above results. The combination can be accomplished using a number of numerical and filtration methods including, but not limited to, Kalman filtration.

[0066] A variety of different sensors could be employed as long as they measure movement relative to the body of the device. Examples of sensors include members of the group consisting of accelerometers, rotational sensors, gyros, magnetometers, and cameras. The user's frame does not have to be stationary. For example, if the user's frame of reference is selected to be the user's forearm, the device would then respond only to movement of the wrist and fingers.

[0067] One skilled in the art will recognize that the commutative property applies to the transformations of the frame of reference that are described in this invention. Consequently, the order of mathematical operations can be altered without materially affecting the invention described herein. In addition, many motion processing algorithms can operate equivalently in any frame of reference, especially when the user frame is chosen to be stationary with a constant orientation.

[0068] In addition to providing ease of use, the framework transformations in accordance with this exemplary embodiment of the present invention can also be used to address other challenges in handheld implementations. For example, if a sensor (such as an accelerometer) is not located exactly at the center of rotation in the body's frame of reference, the measured acceleration will include both the acceleration of the frame and the components of acceleration due to the rotation of the frame. Therefore, the measured acceleration can first be transformed to a different target location within the frame of the device body using the following relationship:

Chord = Accelerometer + ω 'x R + ω x (ω x R) where R is the vector from the accelerometer to the target location, ω is the angular velocity of the reference frame of the body and ω' is the angular acceleration of the reference frame of the body. If the body's frame of reference is constructed such that it is located at R of the accelerometer, then it should have zero angular acceleration effects and can be more easily used to compute the movement of the device in the frame of the user. This compensates for voluntary or inadvertent misalignment between the accelerometer and the center of the body's frame of reference. In addition, the assessment of the gravity vector becomes much easier since there are fewer forces acting on the center of rotation. Then,

Ausuario = Rotate (Body, Q) where Q is the rotation by virtue of which the reference frame of the body is passed to the reference frame of the accelerometer.

ES 2 384 572 T3

Unfortunately, different users have different values ​​for R. For example, one user can use the handheld device by turning the elbow, while another can use the device by turning the wrist. In addition, people have wrists and forearms of different dimensions. For improved usability, this exemplary embodiment of the handheld dynamically computes R and shifts the origin of the body such that it has minimal acceleration components due to angular movement. The exemplary embodiment calculates R by defining R as [Rx, 0, 0] and solving for Rx using and minimizing Arrangement-Rotate [Ag, Q]. Note that there are many numerical methods, including recursive least squares and Kalman filtering, that can perform minimization to compute Rx.

Based on the foregoing, it will be appreciated that the present invention describes various techniques for transforming the detected movement of a handheld device into a frame of reference (such as a body frame of reference). to another frame of reference (such as a user's frame of reference). These transformations can be independent of other transformations associated with the use of the handheld device, such as. ex. the transformation of the detected movement into the movement of a cursor, or they can be combined with them. Furthermore, the transformations according to the present invention can be carried out to transform the detected motion in the three dimensions, to translational motion and rotational motion or any subset thereof, from the perspective of the input side of the motion equation. or on the outlet side. Additionally, the selection of the reference frame to which the detected movement is projected or transformed can be made in a number of different ways. An example given above presents the second frame of reference as a user's frame of reference associated with the tilt of the device, although many other variations are possible. For example, the user can select their desired frame of reference, the setting of which can be stored in the handheld as one of a plurality of user preferences, and can be used to carry out the transformation. The second frame of reference can be selected on the basis of a whole series of techniques. The second frame of reference can be selected on the basis of an explicit command (eg. the selection of a button or user interface) or automatically through user recognition determined by patterns of use of the device, tremor and other biometrics.

[0071] Additionally, despite the fact that some of the embodiments described above operate on data in the speed domain, the present invention is not limited thereto. The projection or transformation according to the present invention can be alternatively or additionally carried out for example on position or acceleration data, and can be for translational movement, for rotational movement, or for both. Likewise, the order of processing is not critical. For example, if the handheld device is used to output gestural commands, projection can be performed first and then gesture can be determined, or gesture can be determined first and then projection can be performed. .

[0072] The exemplary embodiments described above are intended to be illustrative for all purposes, rather than limiting, of the present invention. Thus, the present invention is susceptible to many variations in terms of implementation details that can be deduced from the description contained herein by a person skilled in the art. For example, although the above embodiments describe, among other things, the use of inertial sensors to detect the movement of a device, other types of sensors (such as ultrasonic, magnetic or optical) can be used. instead of or in addition to inertial sensors, in conjunction with the signal processing described above. All such variations and modifications are considered to be within the scope of the present invention as defined in the following claims. No element, act or instruction used in the description of the present application should be considered as critical or essential to the invention unless it is explicitly described as such. Also, in the sense in which it is used herein, the article “a” (or “an”) is considered to include one or more things.