Three-dimensional integrated touch screen input apparatus

The invention relates to a movable in three dimensions, manually operated input and control device. In particular, the present invention relates to an integrated with a base plate of the input device display device with touch screen, with the help of, for example. the basic settings of the input device, such as assignments of triggering functions and / or the sensitivity of the input device in translational or Rotational movements of the input device, can be changed individually.

Since the underlying invention can advantageously be used in the area of ​​control of virtual and / or real objects, the following is intended to refer to the state-of-the-art tools for inputting information that is used to control objects in the area of ​​virtual reality (VR ) Applications are used, will be discussed briefly. Use cases for real objects are, for example. Robot or else electric or Electronic devices in the home when the 3D input device is used in the manner of a remote control. Other applications are, for example. in the automotive sector (adjustment of electric seats, exterior mirrors).

Conventional input systems such as keyboards, mice, trackballs and joysticks are now widely used. They are used to control cursors, mouse pointers, etc., for example, to navigate through a virtual scene or to move virtual objects on the screen or real objects.

With a touch screen, it is possible to point directly at objects that are displayed on the screen with your finger, without the need for additional space-consuming accessories on the desk. Low-resolution touchscreens have 10 to 50 horizontal and vertical positions and use a horizontal and vertical array of infrared light emitting diodes and photosensors to create a grid of invisible light rays just in front of the screen. Touching the screen will interrupt both vertical and horizontal beams of light. Based on this information, the current finger position can be determined.

Another known embodiment of touch-sensitive information input devices is the capacitive coupled touch panel. This provides a resolution of approx. 100 Positions in each direction. When a user touches the conductive coated glass panel of the touch panel with a finger, the current finger position can be detected due to the impedance change. Other high-resolution panels use two minimally spaced, transparent layers. One is coated conductive, the other coated with a resistive material. By the contact pressure of the finger, these two layers touch each other and by measuring the resulting voltage drop then the current finger position can be determined. A lower resolution and cheaper version of this technology uses a grid of fine wires instead of these layers.

Various solutions to the problem of real-time motion control of virtual objects are now available in the prior art, each of which is optimized for a particular application. Each of these solutions therefore has certain limitations. In order to explain some of the most important of these solutions, it is necessary to briefly discuss their main features.

One possibility for real-time motion control of virtual objects has recently come about through the development of computer input devices that allow the simultaneous input of drive signals of multiple, independent degrees of freedom. The possibilities thereby created far exceed those that exist, for example, when using a mouse, the only two-dimensional (eg on the shelf of a desk) can be controlled. Although it is also known to provide a mouse with additional switches, for example, these switches have the disadvantage that they do not allow the input of analog data, but rather are limited to binary data (on / off).

Various input devices are known from the prior art, which can generate analog drive signals with different, independent degrees of freedom, each of these analog signals can thus be used as a parameter value in a control of virtual objects. Such manually controllable input systems, which permit navigation in three dimensions, are successfully used today in a number of very different technical fields of application.

For example, US-A-5,757,360 discloses an egg-shaped computer input device that can be moved freely in space by a user's hand, determines its current positions, directions of movement, speeds and accelerations, and transmits these kinematic data wirelessly to a computer. In this case, an analogous movement sequence in the form of a movement pattern is identified, from which movement commands are derived and converted into an animated graphic representation. The motion patterns are automatically detected using a pattern recognition algorithm. In addition, control commands are generated. The disadvantage of this method is that it is not freely specifiable, since movement sequences of the user, which are detected analogously by the input device, corresponding motion sequences of stored motion sequences of an animated graphic representation are assigned and can only be displayed as such.

Input devices that have manually operated force-moment sensors are known for example from the patents DE 36 11 336 C2, DE 37 64 287 and EP 0 979 990 A2.

From the last-mentioned European patent EP 0 979 990 A2 is known such a force-moment sensor for controlling operating elements of a real or virtual mixing or To use control panels, for example, to create and design novel color, light and / or sound compositions. In this case, the intuitive spatial control in three translatory and three rotational degrees of freedom can advantageously be transferred to a continuous spatial mixing or control of a large number of optical and / or acoustic parameters. To control a pressure is exerted on the user interface of the input device, thereby generating a pulse which is detected by means of the force-moment sensor and converted into a consisting of a force and a moment vector vector pair. If certain characteristic impulse specifications are met, for example, an object-specific control operation and / or a technical function can be triggered by switching to an activation state or be terminated by switching to a deactivation state again.

From this document is also known to attach said force-moment sensor as a control element of a 3D input device laterally on a touch screen, so that the longitudinal axis of the control element is parallel to the display and control surface of the touch screen. However, this has the disadvantage that the viewing direction on the touchscreen does not coincide with the longitudinal axis of the operating element. This has the consequence that the hand-eye coordination of the user is difficult because the directions of control movements of the control element and displayed object movements on the screen in unfavorable axis occupation of the control element do not match.

From the G 298 04 023 a hemispherical input device with push buttons for controlling synthesizers or MIDI controllers is known.

OBJECT OF THE PRESENT INVENTION

Based on the above-mentioned prior art, the present invention addresses the problem of providing a comfortable programmable input device that allows the user to interactive, intuitive motion control virtual objects in real time while improving the coordination of the hand and eyes. In particular, the programming of this input device should be simplified without the need to program via menus of a computer-controlled configuration program.

This object is achieved by the features of the independent claims. Advantageous embodiments which further develop the idea of ​​the invention are defined in the dependent claims.

SUMMARY OF THE PRESENT INVENTION

In the context of the underlying invention, the man-machine interface is represented by a 3D input device. It should be noted that this 3D input device is only one example of an input device for generating drive signals. In principle, any other type of input device is also suitable. However, the inventive 3D input device has the great advantage that the input of drive signals with different degrees of freedom can be done in a particularly intuitive manner.

According to the preferred first embodiment of the underlying invention, a 3D input device for transmitting drive data to electronic or electrical devices is provided. This 3D input device has according to the invention<ul id="ul0001" list-style="dash" compact="compact"><li>an operating part which is designed in such a way that it can be manipulated directly or indirectly by at least one finger or a hand of the user,</li><li>a base plate with respect to which the operating part is movably mounted and</li></ul> a display device with a touch screen. The relative movement between the control panel and the base plate is evaluated to generate the control data. The touch screen is integrated into the top of the base plate, which is a compact and user-friendly arrangement.

According to a second exemplary embodiment of the underlying invention, a 3D input device is provided for the transmission of drive data to electronic or electrical devices, comprising the following components:<ul id="ul0002" list-style="dash" compact="compact"><li>an operating part that is designed such that it can be manipulated directly or indirectly by at least one finger or with one hand of the user,</li><li>a base plate with respect to which the operating part is movably mounted and</li></ul> a display device with a touch screen. The relative movement between the control panel and the base plate is evaluated to generate the control data. The touch screen is integrated into the top of the control panel. This represents a particularly compact embodiment.

According to a further aspect of the invention, a 3D input device is provided for the transmission of drive data to electronic or electrical devices, comprising the following components:<ul id="ul0003" list-style="dash" compact="compact"><li>an operating part that is designed such that it can be manipulated directly or indirectly by at least one finger or with one hand of the user,</li><li>a base plate with respect to which the operating part is movably mounted and</li><li>a display device with a touch screen, wherein the touch screen is arranged substantially perpendicular to the longitudinal axis of the control panel. The relative movement between the control panel and the base plate is evaluated to generate the control data.</li></ul> Thus, advantageously, the viewing direction on the touchscreen coincides with the longitudinal axis of the operating part, which allows a simplified hand-eye coordination.

The device may have a processor which is functionally coupled to the touch screen and the control panel, so that by means of an input on the touch screen parameters of the 3D input device, such as sensitivity, axis assignment, function key assignment etc. are adjustable.

In addition, according to the invention, the sensitivity, the response thresholds and / or the response times of the 3D input device for movements with up to three translational and / or three rotational degrees of freedom can be optionally set by manual inputs via the touch screen of the display device.

By manual inputs via the touch screen of the display device, an assignment of optional technical functions to the degrees of freedom of the 3D input device can also be set.

The base plate can be used to rest on a work surface.

In a special application of the present invention it can be provided that the device has an interface for the wireless transmission of control data to an electrically operated device. This interface makes it possible to use the input device in the manner of a remote control.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features, features, advantages and advantages of the underlying invention will become apparent from the subordinate dependent claims and from the following description of the preferred embodiments of the invention, which are illustrated in the following drawings. Herein show:<dl id="dl0001"><dt>FIG. 1a</dt><dd>10 is a simplified block diagram 100a according to the preferred first embodiment of the underlying invention, in which a display device with touch screen 107 is integrated in the upper side of the base plate 106,</dd><dt>FIG. 1b</dt><dd>5 is a simplified block diagram 100b according to a second exemplary embodiment of the underlying invention, in which a display device with touchscreen 107 is integrated in the upper side of the operating part 104 of the 3D input device 102,</dd><dt>FIG. 2a</dt><dd>a detailed block diagram 200a illustrating the components of the 3D input device 102 and the exchanged between these components signals 110a + b and 112a + b according to the preferred first embodiment of the underlying invention, wherein a display device with touch screen 107 in the top of the base plate 106th is integrated,</dd><dt>FIG. 2 B</dt><dd>a detailed block diagram 200b to illustrate the components of the 3D input device 102 and the exchanged between these components signals 110a + b and 112a + b according to the second embodiment of the underlying invention, in which a display device with touch screen 107 in the top of the operating part 104 of 3D input device 102 is integrated,</dd><dt>FIG. 3a</dt><dd>a three-dimensional view 300a of the 3D input device 102 according to the preferred first embodiment of the underlying invention, in which a display device with touch screen 107 in the top of the base plate 106 of the 3D input device 102 is integrated, with a coordinate system in which the six degrees of freedom x , y, z and φ_x, φ_y, φ_z of the 3D input device and their first and second time derivatives ẋ, ẏ, ż and φ̇_x, φ̇_y, φ̇_z such as <img file="EP1513056A2_D0001.tif" />, Ÿ,<img file="EP1513056A2_D0002.tif" /> and <img file="EP1513056A2_D0003.tif" />. <img file="EP1513056A2_D0004.tif" />. <img file="EP1513056A2_D0005.tif" /> are drawn in,</dd><dt>FIG. 3b</dt><dd>a three-dimensional view 300b of the 3D input device 102 according to the second embodiment of the underlying invention, in which a display device with touch screen 107 is integrated into the upper side of the operating part 104 of the 3D input device 102, with a coordinate system in which the six degrees of freedom x, y, z and φ_x, φ_y, φ_z of the 3D input device and their first and second time derivatives ẋ, ẏ, ż and φ̇_x, φ̇_y , φ̇_z such as <img file="EP1513056A2_D0006.tif" />, Ÿ,<img file="EP1513056A2_D0007.tif" /> and <img file="EP1513056A2_D0008.tif" />. <img file="EP1513056A2_D0009.tif" />. <img file="EP1513056A2_D0010.tif" /> are drawn in,</dd><dt>FIG. 4</dt><dd>a window 400 of the display device 107 with integrated touch screen for adjusting the sensitivity levels of the 3D input device 102 for translational movements of virtual objects in the x-, y- or z-direction and for rotational movements of virtual objects in φ_x-, φ_y- or φ_zDirection, whereby a distinction can be made between linear and non-linear sensitivity behavior,</dd><dt>FIG. 5</dt><dd>a window 500 of the display device 107 with integrated touch screen for setting the thresholds of the 3D input device 102 for translational movements of virtual objects in the x-, y- and z-direction, for rotational movements of virtual objects in φ_x- <i>φ</i>_y- or φ_zDirection and for setting the response times of the 3D input device 102 for translation or. Rotational movements and</dd><dt>FIG. 6</dt><dd>a window 600 of the display device 107 with integrated touch screen for setting the key assignment for the nine keys of the 3D input device 102 and the Quicktip® softkey with functions from a selection window by pressing an information and an activation button.</dd></dl>

DETAILED DESCRIPTION OF THE INVENTION

In the following, the functions of the assemblies included in the preferred embodiment of the present invention as illustrated in Figs. 1a to 6 will be described in more detail. In this case, the construction and the mechanical components of a 3D input device with touch screen according to two embodiments of the present invention will be explained first.

The 3D input device according to the invention comprises the following components:<ul id="ul0004" list-style="dash"><li>a control panel 104 that may be manipulated indirectly (ie without direct access) or directly (directly accessed) with at least one user's finger or hand;</li><li>a base plate 106 on which the operating part 104 is movably mounted in three axes, at any time t<maths id="math0001" num=""><img file="EP1513056A2_D0011.tif" /></maths> and<maths id="math0002" num=""><img file="EP1513056A2_D0012.tif" /></maths> with components F_x(t), F_y(t), F_z (t), M_x (t), M_y(t) and M_z (t) in the direction of the unit vectors<img file="EP1513056A2_D0013.tif" />. <img file="EP1513056A2_D0014.tif" /> and <img file="EP1513056A2_D0015.tif" /> a three-dimensional coordinate system with the axes x, y and z record as well</li><li>Function keys 106a that can be freely programmed by the user.</li></ul>

Movement signals of the user with up to three translational degrees of freedom x, y and z and / or up to three rotational degrees of freedom φ_x, φ_y or φ_z are interpreted as control signals 114 from the control panel 104 to the base plate 106th

FIG. 1 a shows a simplified block diagram 100 a according to the preferred first exemplary embodiment of the underlying invention, in which a display device with a touch screen 107 is integrated in the upper side of the base plate 106. In contrast, FIG. 1 b shows a simplified block diagram 100 b according to a second exemplary embodiment of the underlying invention, in which a display device with touchscreen 107 is integrated into the upper side of the operating part 104 of the 3D input device 102. With the help of the touch screen, a programming of the function keys 106a can be made possible.

Referring to the two block diagrams 200a and 200b in FIGS. 2a and 2b respectively, the components of the 3D input device 102 and the signals exchanged between these components will be explained in more detail below according to the preferred first and second exemplary embodiments of the underlying invention.

FIG. 2 a shows a detailed block diagram 200 a for illustrating the components of the 3D input device 102 and the signals 110 a + b and 112 a + b exchanged between these components according to the preferred first exemplary embodiment of the underlying invention, in which a touch screen display device 107 in FIG the top of the base plate 106 is integrated.

FIG. 2b shows a detailed block diagram 200b for illustrating the components of the 3D input device 102 and the signals 110a + b and 112a + b exchanged between these components according to the second exemplary embodiment of the underlying invention, in which a display device with a touch screen 107 is placed in the top side of the display Operating part 104 of the 3D input device 102 is integrated.

In order to enable communication between the operating part 104 and the base plate 106, both components are connected to one another via an input / output interface for exchanging activation signals 114, which is not shown in FIGS. 2a and 2b. The 3D input device 102 also has an interface 116 to a computer. Programs entered via the touch screen input device 107 are sent as programming signals 110b to a microprocessor 106b integrated in the base plate 106, and arrive as drive signals 112b to the program signal input of the function keys 106a. In the reverse direction, data signals 112a are sent from the data signal output of the function keys 106a to the microprocessor 106b, as far as input has been made by the user. They arrive as data signals 110a to the signal input of the display device with touch screen 107 to display or to be graphically visualized.

When appropriately controlled by the user, this 3D input device 102 is capable of generating drive signals 108 for six independent degrees of freedom. These include three translational degrees of freedom, hereafter referred to as x, y and z, and three rotational degrees of freedom in the following as φ_x, φ_y and φ_z be designated. The variables x, y and z denote the orthogonal axes of a three-dimensional Cartesian coordinate system. If these variables are combined in vector notation, then at every discrete point in time n (after omission of the units) results:<ul id="ul0005" list-style="none" compact="compact"><li>the location vector<maths id="math0003" num=""><img file="EP1513056A2_D0016.tif" /></maths> and</li><li>the direction of rotation vector<maths id="math0004" num=""><img file="EP1513056A2_D0017.tif" /></maths></li></ul>

Of course, with the help of key controls or switches relatively easy even more degrees of freedom can be added. It should be noted that switches or keys generally generate binary drive signals (on / off), whereas the above-mentioned three translatory degrees of freedom x, y and z or the three rotational degrees of freedom φ_x, φ_y or φ_z can each give analog control signals, which then, for example, in byte-wise coding in 2^8th = 256 levels are available as digital signals for further processing.

Since the three translational and three rotational degrees of freedom x, y, z, φ_x, φ_y or φ_z as "analog signals" or According to the present invention, the temporal variation of these drive signals 108 can be evaluated by the freely specifiable real-time controller 102 for animated graphics, video and / or audio sequences described in more detail below. In particular, it is thus possible for every discrete point in time n (after omitting the units) to have the three-dimensional vectors for the speed<maths id="math0005" num=""><img file="EP1513056A2_D0018.tif" /></maths> the acceleration<maths id="math0006" num=""><img file="EP1513056A2_D0019.tif" /></maths> the angular velocity<maths id="math0007" num=""><img file="EP1513056A2_D0020.tif" /></maths> such as the angular acceleration<maths id="math0008" num=""><img file="EP1513056A2_D0021.tif" /></maths> the degrees of freedom x, y, z, φ_x, φ_y or φ_z to detect and, where appropriate, to be used as further degrees of freedom independent of the absolute value of the respective drive signals 108. In this case, no further sensors (eg speed sensors or acceleration sensors) are necessary on the part of the input device 102.

FIG. 3 a shows a three-dimensional view 300 a of the 3D input device 102 according to the preferred first exemplary embodiment of the underlying invention with a 3D coordinate system in which the six degrees of freedom<maths id="math0009" num=""><math display="block"><mrow><msub><mrow><mtext>x, y, z [m] and φ</mtext></mrow><mrow><mtext>x</mtext></mrow></msub><msub><mrow><mtext>, φ</mtext></mrow><mrow><mtext>y</mtext></mrow></msub><msub><mrow><mtext>, φ</mtext></mrow><mrow><mtext>z</mtext></mrow></msub><mtext> [wheel]</mtext></mrow></math><img file="EP1513056A2_D0022.tif" /></maths> of the 3D input device 102 and their first and second time derivatives    ẋ, ẏ, ż [m · s^-1] and φ̇_x, φ̇_y , φ̇_z [Rad · s^-1] such as    <img file="EP1513056A2_D0023.tif" />, Ÿ,<img file="EP1513056A2_D0024.tif" /> [M · s^-2] and <img file="EP1513056A2_D0025.tif" />. <img file="EP1513056A2_D0026.tif" />. <img file="EP1513056A2_D0027.tif" /> [Rad · s^-2] are drawn. The outlined 3D input device 102 is a modified model of the SpaceMouse® "Classic" from LogiCad3D GmbH with a total of nine freely programmable function keys, in which according to the invention a touch screen 107 is integrated into the upper side of the base plate 106.

In an analogous manner, FIG. 3b shows a three-dimensional view 300b of the 3D input device 102 according to the second exemplary embodiment of the underlying invention with a 3D coordinate system in which the six degrees of freedom of the 3D input device 102 and their first and second time derivatives are plotted. Again, the outlined 3D input device 102 is a modified model of the SpaceMouse® "Classic" from LogiCad3D GmbH with a total of nine freely programmable function keys, in which according to the invention a touch screen 107 is integrated in the upper side of the operating part 104.

By manual inputs via the touch screen of the display device 107, it is possible according to the invention, the sensitivity of the 3D input device for movements with up to three translational and / or three rotational degrees of freedom individually set in each 600 levels. In the same way, the response thresholds and the response times of the 3D input device 102 for translatory and / or rotational movements of virtual objects can be adjusted individually with the aid of the touch screen of the display device 107 according to the invention.

FIG. 4 shows a window 400 of the display device 107 with integrated touchscreen for adjusting the sensitivity of the 3D input device 102 for translational movements of virtual objects in the x, y and z directions as well as for rotational movements of virtual objects in φ_x-, φ_y- or φ_z-Direction. The sensitivity can be changed by pressing the appropriate softkeys and sliders and adjusted to the user's mode of operation. With the help of two additional softkeys, a distinction can be made between linear and non-linear sensitivity behavior. In total, 600 sensitivity levels are available to set the sensitivity.

FIG. 5 shows a window 500 of the display device 107 with integrated touchscreen for setting the response thresholds of the 3D input device 102 for translational movements of virtual objects in the x, y and z directions as well as rotational movements of virtual objects in φ_x-, φ_y- or φ_zDirection outlined. The thresholds can be changed by pressing the appropriate softkeys and sliders and adjusted to the user's mode of operation. In addition, two further softkeys are provided in this window, with the aid of which an adjustment of the response times of the 3D input device 102 for translational or rotational movements of virtual objects can be made individually.

FIG. 6 shows a window 600 of the display device 107 with integrated touchscreen for setting the key assignment for the nine keys of the 3D input device 102 and the Quicktip® softkey. The assignment of technical functions to specific function keys 106a of the 3D input device 102 can be effected, for example, by actuating the softkeys referring to the numbers of the function keys 106a of the 3D input device 102 and a selection of functions from a scrollable menu. An assignment is considered programmed if the activation key was pressed. With the aid of the information key, the user can get further information on the properties of selected functions which, if desired, are displayed in a further window of the display device 107.

The meaning of the symbols provided with reference symbols in FIGS. 1a to 6 can be found in the attached list of reference numerals.

LIST OF REFERENCE NUMBERS

<tables id="tabl0001" num="0001"><img file="EP1513056A2_D0028.tif" /></tables><tables id="tabl0002" num="0002"><img file="EP1513056A2_D0029.tif" /></tables><tables id="tabl0003" num="0003"><img file="EP1513056A2_D0030.tif" /></tables>