Orientation free user interface

The present invention relates to a method of changing the orientation of a user interface and a display system that controls an orientable user interface.

Computer system utilities can be enhanced by providing a better user interface. The user interface of computer systems has evolved significantly since the first widespread availability of personal computers (PCs). Early PCs used rather primitive user input devices, such as serial mice, and provided only monochrome displays. However, all of the huge improvements in microprocessors, available memory, and programming functionality have contributed to advances in user interface design and the development of user-friendly graphics operating systems and hardware.

One particular area of progress in user interface technology concerns interactive display systems. Such systems typically include enclosures, display surfaces, various image projection and user input detection devices, and computers. The interactive display system may include or be coupled to a general purpose computing device. A major advantage of interactive display systems is that they allow direct user interaction with the display surface. The user can supply input to this type of system by directly touching the interactive display surface with one or more fingers, or by making a gesture just above the interactive display surface. ..

<p> However, interactive display systems are sometimes constrained to implement a graphical user interface (GUI) similar to that used in applications designed to run on traditional personal computers (PCs). May suffer species restrictions and disadvantages. For example, a user of an interactive display system can interact with a horizontal display surface from any side. By its very nature, the horizontal interactive display surface has "top" and "bottom" and does not resemble a traditional vertical display intended to be seen by a person sitting in front of a traditional display. .. Unlike traditional vertically oriented displays, users placed on either side of an interactive display clearly have a graphical user interface oriented towards the user in order to have a similar user experience. prefer. Thus, a graphical user interface oriented for a user on one side of the horizontal interactive display surface will look opposite to the user on the other side of the interactive display surface, and vice versa. Therefore, there is now an incentive to develop solutions to the above and other prior art limitations that hinder the user's enjoyment of generally horizontal interactive display surfaces. It is clearly preferable to provide a dynamically orientation-free user interface for use with applications running on interactive display systems.</p><p><patcit num="1"><text>U.S. Patent Application No. 10 / 814,577</text></patcit><patcit num="2"><text>U.S. Patent Application No. 10 / 814,761</text></patcit></p>

<p> Various embodiments are described below that allow the generation and display of user interfaces with dynamically orientable user input controls. Specifically, the user interface can be displayed on the interactive display via a computer-implemented method, so that the user interface is provided by users placed at different locations near the periphery of the interactive display. Oriented appropriately as seen. Further details of exemplary embodiments are presented below. This embodiment includes selecting a portion of the main graphical user interface (GUI). An independent, dynamically orientable GUI is then generated within the selected portion. In this embodiment, the dynamically orientable GUI can provide one or more user input controls, which are based on the input provided by the user. It is possible to be oriented within the selected part of. This user input can be provided by user interaction with an orientable GUI provided on the interactive display.</p><p> This "means for solving the problem" is provided to introduce in a simplified form a few concepts that are further explained in detail in "The best form for carrying out the invention" below. However, this "means for solving a problem", even if intended to identify the main or essential features of the claimed subject, can help determine the scope of the claimed subject. It is not intended to be used.</p><p> Various aspects and accompanying benefits of one or more exemplary embodiments and modifications thereof will be better understood by reference to the detailed description that follows when interpreted with the accompanying drawings. Therefore, it will be easier to understand.</p>

<u style="single">The drawings and disclosed embodiments are not limited.</u> An exemplary embodiment is shown in the referenced figure of the drawing. The embodiments and drawings disclosed herein are intended to be considered exemplary rather than restrictive. In addition, in the appended claims, the list of alternatives includes the conjunctive words "and", "to", or "and" before the phrase "at least one" or the phrase "one of". When used, the intended meanings of these "and", "and", or "and" words correspond to the connecting words "or", "or", or "or".

<u style="single">Illustrative computing system</u> FIG. 1 is a functional block diagram of an exemplary computing system and / or computer server that supplies digital media to connected client computing devices, such as an interactive display table or similar computing system.

Subsequent discussions are intended to provide a short general description of a suitable computing environment in which a method can be implemented. In addition, subsequent discussions provide context for implementing computer-executable instructions such as program modules with computing systems. In general, a program module includes routines, programs, objects, components, data structures, etc. that perform a particular task or perform a particular abstract data type. We include multiprocessor systems, mainframe computers, personal computers, processor-controlled consumer electronics, personal digital assistants (PDAs) (probably not when used as servers for digital media content), and others. Will recognize that the computing system configuration of is applicable. Other embodiments include a distributed computing environment in which tasks are performed by remote processing devices linked over a communication network. In a distributed computing environment, program modules can be located on both local and remote memory storage devices.

With reference to Figure 1, an exemplary system suitable for implementing various methods is shown. The system includes a general purpose computing device in the form of a conventional personal computer (PC) 20 provided with a processing unit 21, system memory 22, and system bus 23. The system bus combines various system components, including system memory, into processing unit 21, and includes multiple types of buses, including memory buses or memory controllers, peripheral buses, and local buses that use either of the various bus architectures. It can be any of the structures. System memory includes read-only memory (ROM) 24 and random access memory (RAM) 25.

ROM 24 contains a basic I / O system 26 (BIOS) that contains basic routines that allow information to be transferred between elements within PC 20, such as during system startup. The PC 20 also includes a hard disk drive 27 that reads from and writes to a hard disk (not shown), a magnetic disk drive 28 that reads from or writes to a removable magnetic disk 29, and a compact disk read-only memory (CD-ROM). Or includes an optical disk drive 30 that reads from or writes to a removable optical disk 31 such as another optical medium. The hard disk drive 27, the magnetic disk drive 28, and the optical disk drive 30 are connected to the system bus 23 by the hard disk drive interface 32, the magnetic disk drive interface 33, and the optical disk drive interface 34, respectively. These drives and their associated computer-readable media are PCs. Provides non-volatile storage of 20 computer-readable machine instructions, data structures, program modules, and other data. An exemplary environment described uses a hard disk 27, a removable magnetic disk 29, and a removable optical disk 31, but with a magnetic cassette, flash memory card, digital video disk (DVD), Bernoulli cartridge, RAM, ROM, and similar. Those skilled in the art will recognize that other types of computer-readable media that can store computer-accessible data and machine instructions, such as objects, can also be used.

Store multiple program modules, including operating system 35, one or more application programs 36, other program modules 37, and program data 38, on a hard disk 27, magnetic disk 29, optical disk 31, ROM 24, or RAM 25. be able to. The user can use the PC via an input device such as keyboard 40 and pointing device 42. You can enter commands and information within 20 to provide control input. The pointing device 42 may include a mouse, stylus, wireless remote control, or other pointer, but such conventional pointing devices may be omitted in connection with the embodiments currently described. This is because the user can use the interactive display system for input and control. As used in this description, the term "mouse" is intended to include all pointing devices that are useful in controlling the position of a cursor on the screen. Other input devices (not shown) can include microphones, joysticks, tactile joysticks, yokes, foot pedals, gamepads, satellite dishes, scanners, or the like. Also, PC The 20 may include a Bluetooth wireless interface or other wireless interface for communication with other interface devices such as printers or the interactive display table described in detail below. The above and other input / output (I / O) devices can be connected to the processing unit 21 via the I / O interface 46 coupled to the system bus 23. The clause "I / O interface" specifically refers to serial ports, parallel ports, game ports, keyboard ports, and / or universal serial. It is intended to include each interface used for bus (USB). The system bus 23 can also be connected to a camera interface (not shown), which receives signals from the digital video camera contained within the interactive display 60, as described in more detail below. Combined with the interactive display 60. A digital video camera can instead be coupled to a suitable serial I / O port, such as a USB port. The system bus 23 can also be connected to the light source in the interactive display via the I / O interface 46 or another interface to provide control signals to the light source, as described in more detail below. In addition, the system bus 23 can be connected to a photodetector in an interactive display system via the I / O interface 46 or another interface to receive user input. In some cases, the monitor 47 can be connected to the system bus 23 via a suitable interface such as a video adapter 48, but the monitor is likely to be omitted. This is because the interactive display system described below can provide a richer display and can also interact with the user to enter information and control software applications, and is therefore preferably a video adapter. Because it is bound to. In general, a PC can also be coupled to other peripheral output devices (not shown), such as speakers (via a sound card or other audio interface (not shown)) and a printer.

Some methods described in detail below can be practiced on a single machine, but PC 20 is networked using a logical connection to one or more remote computers, such as remote computer 49. It can also operate in a computerized environment. The remote computer 49 may be another PC, a server (which can be configured much like PC 20), a router, a network PC, a peer device, or a satellite or other common network node (none of which is shown). ), Which usually contain many or all of the elements described above in relation to the PC 20, but only the external memory storage device 50 is illustrated in Figure 1. The logical connections shown in Figure 1 include a local area network (LAN) 51 and a wide area network (WAN) 52. Such networking environments are common in offices, enterprise-wide computer networks, intranets, and the Internet.

When used in a LAN networking environment, PC 20 is connected to LAN 51 via a network interface or network adapter 53. When used in a WAN networking environment, the PC 20 typically has a modem 54 or cable modem, digital subscriber line (DSL) interface, or integrated services digital to establish communication over WAN 52, such as the Internet. Includes other means such as network (ISDN) interfaces. The modem 54, which can be internal or external, is connected to the system bus 23 or is coupled to the bus via the I / O device interface 46 or serial port. In a networked environment, program modules used by PC 20 or parts thereof can be stored in remote memory storage devices. It should be appreciated that the network connections shown are exemplary and other means of establishing communication links between computers, such as wireless communications and broadband network links, can be used.

<u style="single">Illustrative interactive surface</u> FIG. 2 shows an exemplary interactive display table 60 that includes a PC 20 in frame 62 and acts as both an optical input for a computer and a video display device. The illustrated embodiment is a cut-out view of one embodiment of the interactive display table 60. In the embodiment shown in FIG. 2, the rays 82a-82c used to display text and graphic images are shown using dotted lines and are in contact with or immediately after the interactive display surface 64 of the interactive display table 60. The infrared (IR) rays used to sense the object above are shown using dashed lines. The perimeter of the table surface is useful for supporting the user's arm or other object, including an object that can be used to interact with the graphic image or virtual environment displayed on the interactive display surface 64.

Light source 66 may include any of a variety of light emitting devices, such as light emitting diodes (LEDs), laser diodes, and other suitable light sources that can be driven to scan in two orthogonal dimensions, the X and Y directions. it can. Scanning mechanisms such as rotating mirrors, galvano mirrors, or other well-known scanning mechanisms commonly used to make surface raster scans with light beams are used with light source 66 for each of the other light sources described below. can do. In general, the light source 66 is configured to emit light having a wavelength in the infrared (IR) spectrum and is therefore not visible to the human eye. However, light of any wavelength that is invisible to the human eye can be used to avoid interference with the display of the visible image provided by the interactive display surface 64. The light source 66 can be mounted at any position inside the frame 62, depending on the particular light source used. The light produced by the light source 66 is directed upwards toward the bottom of the interactive display surface 64, as indicated by the dashed lines 78a, 78b, and 78c. The light emitted from the light source 66 contacts or abuts on the interactive display surface 64 after passing through the translucid layer 64a of the table, including a sheet of vellum or other suitable translucent material that has the property of diffusing light. Reflected from any object adjacent to.

As used in this description and in the appended claims, the term "in the immediate vicinity of" means that the phrase touches or interacts with the surface of an interactive display, depending on factors such as the reflectance of the object. Used with the intent to include both objects that are either short distances from the surface of the type display, eg, up to 3 cm or more. Although only one light source 66 is shown, it should be appreciated that multiple such light sources can be mounted at isolated locations inside the frame 62 to provide uniform illumination of the interactive display surface. .. The light produced by the light source 66 exits through the table surface without illuminating the object at all as indicated by the dashed line 78a, illuminates the object in contact with the table surface as indicated by the dashed line 78b, and / or is indicated by the dashed line 78c. It can be either a short distance on the surface of the interactive display (ie, in close proximity to it), but illuminates an object that does not come into contact with it.

Objects on the interactive display surface 64 are the "touch" object 76a, which "touches" or at least partially touches the display surface, and the "hover" object, which is close to the interactive display surface but does not actually touch it. Including 76b. Thus, both touch and hover objects can be "close" to the display surface when the term "proximity" is used in subsequent discussions. Reflected by an object as it approaches the top of the interactive display surface 64 as a result of the use of a translucent layer 64a underneath the interactive display surface to diffuse light that passes through the interactive display surface. The amount of IR light produced increases to the maximum level when the object is actually in contact with the display surface.

As shown in FIG. 2, the photodetector 68 is placed (ie, in close proximity to) a "touch" object or "hover" below the interactive display surface 64 and above the interactive display surface. It is mounted on the frame 62 at a suitable position to detect the IR light reflected from the object. In general, the photodetector 68 can be any photodetector suitable for detecting light reflected from an object in contact with or in close proximity to the interactive display surface 64. For example, the photodetector 68 is an area CMOS (area). It can be a CMOS) sensor or an area charge-coupled device (CCD) sensor. The embodiment shown in FIG. 2 shows one photodetector 68, but multiple photodetectors 68 can be used within the interactive display 60. The photodetector 68 can be equipped with an IR pass filter 86a that allows only IR light to pass through and blocks ambient visible light traveling along the dotted line 84a through the interactive display surface 64. In this embodiment, a baffle 79 is placed between the scanning light source 66 and the photodetector 68 to prevent IR light emitted directly from the scanning light source 66 from entering the photodetector 68. This is because the photodetector 68 preferably produces an output signal that responds only to IR light reflected from an object close to the interactive display surface 64. The photodetector 68 also includes all IR light contained in the ambient light passing through the interactive display surface 64 from above into the interactive display, including the ambient IR light also traveling along the path indicated by the dotted line 84a. It is clear that it will react.

IR light reflected from an object in contact with or above the table surface passes through the translucent layer 64a, through the IR pass filter 86a, to the photodetector 68, as indicated by (a) dashed lines 80a and 80b. May be reflected back, or (b) reflected or absorbed by other inner surfaces within the interactive display 60 without entering the photodetector 68, as indicated by the dashed line 80c. There is.

The translucent layer 64a diffuses both incident IR light and reflected IR light. Therefore, as explained above, a "hover" object, such as a hover object 76b closer to the interactive display surface 64, returns more IR light to the photodetector 68 than an object of the same reflectance farther from the display surface. reflect. The photodetector 68 senses the IR light reflected from the "touch" and "hover" objects in its operating field and creates a detection signal corresponding to the reflected IR light received by the photodetector 68. .. This detection signal is processed by a PC to determine the position of each such object and, in some cases, other parameters such as the size, orientation, shape, and trajectory of that object. Entered in 20. Note that some objects, such as the user's forearm, may be on the table while another, such as the user's fingers, may be in contact with the display surface. In addition, other parameters related to the object can be detected. For example, an object can include, on its bottom surface, a coded identifier, such as an IR light reflection pattern or barcode, that is specific to the object or specific to the class of related object of which the object is a member. .. Therefore, the detection signals from one or more photodetectors 68 to detect each such particular object and / or object depending on the IR light reflected from the object and / or from the reflection pattern. It can also be used to determine other parameters related to.

Accordingly, embodiments use IR light reflected from an object to detect the identifiable properties of the object, thereby determining the position of the object and / or its position relative to the interactive display surface 64 and other information. It works to recognize. Details of the logical steps performed to so detect and identify an object, its orientation, and other parameters are all described in Patent Document 1, entitled "Identification Of Object," filed March 31, 2004. Described in a patent application transferred to the same assignee as the present application, including "On Interactive Display Surface By Identifying Coded Pattern" and Patent Document 2, entitled "Determining Connectedness And Offset Of 3D Objects Relative To An Interactive Surface". There is. The disclosures and drawings of these two patent applications are specifically incorporated herein by reference (as background information), but are not considered essential to enable the novel approach claimed herein.

The PC 20 can be integrated with the interactive display table 60, as shown in the second embodiment, or the interactive display table 60, as shown in the third embodiment. Can be external. In FIG. 3, the interactive display table 60'is connected to an external PC 20 (including the optional monitor 47 as mentioned above) via a data cable 63. In the alternative, the PC 20 can be connected to the interactive display table 60'via a wireless link (ie, WiFi or any other suitable wireless signal link). As also shown in this figure, the set of orthogonal X and Y axes and the origin indicated by "0" are related to the interactive display surface 64. An exemplary projected image 390 is also illustrated as the word "image" on the display surface 64. It should be appreciated that although not shown separately, multiple coordinate positions along each orthogonal axis can be used to specify any position on the interactive display surface 64.

The interactive display table 60'can be used as an external PC 20 (as shown in Figure 3) or some other type of external computing device, such as a set-top box, video game, laptop computer, or media computer (not shown). When connected, the interactive display table 60'contains input / output devices. Power for the interactive display table 60'is supplied via a power supply lead 61, which is coupled to a conventional alternating current (AC) source (not shown). Data cable 63 connects to the interactive display table 60', but connects to a USB 2.0 port on the PC 20, an Institute of Electrical and Electronics Engineers (IEEE) 1394 (or Firewire) port, or an Ethernet® port. can do. As the speed of wireless connections continues to improve, interactive display tables 60'are placed on PCs via such high-speed wireless connections, or via some other suitable wired or wireless data communication link. It is also intended to be able to connect to computing devices such as the 20. Whether included internally or externally as an integral part of the interactive display system, the PC 20 executes algorithms to process digital images from the digital video camera 68, making the interactive display table more intuitive. User Interface Running software applications designed to take advantage of functionality, while still interactive display table I / O not specifically designed to take advantage of such functionality. Run other software applications that take full advantage of their functionality. As a further alternative, the interactive display system can be coupled to an external computing device, but can include an internal computing device to perform image processing and other tasks that should not be done by an external PC.

An important and powerful feature of the interactive display table 60 or 60'(ie, any of the interactive display table embodiments described above) is an object or object that rests on the display surface, such as object 76a. Display graphic images or virtual environments of games or other software applications by identifying objects (or their characteristics) floating just above the display surface, such as 76b, and visible graphics on the interactive display surface 64. The ability to enable user interaction with an image or virtual environment.

Seeing FIG. 2 again, the interactive display table 60 can include a video projector 70 used to display graphic images, virtual environments or text information on the interactive display surface 64. This video projector can be of liquid crystal display (LCD) or digital light processor (DLP) type or liquid crystal on, for example with a resolution of at least 640 x 480 pixels. It can be a silicon (LCoS) display type. The IR cut filter 86b can be mounted in front of the projector lens of the video projector 70 to allow IR light to interfere with IR light reflected from objects (s) in contact with or above the interactive display surface 64. It is possible to prevent the IR light emitted by the video projector from entering the inside of the interactive display table housing. The video projector 70 projects light toward the first mirror assembly 72a along the dotted path 82a. The first mirror assembly 72a reflects the projected light from the dotted path 82a received from the video projector 70 along the dotted path 82b through the transparent opening 90a in the frame 62, resulting in the reflected projection. The emitted light will be incident on the second mirror assembly 72b. The second mirror assembly 72b reflects the light from the dotted path 82b to the translucidum layer 64a along the dotted path 82c, which is in the focal point of the projector lens, resulting in the projected image. It becomes visible and in focus on the interactive display surface 64 for viewing.

Alignment devices 74a and 74b are provided with threaded rods and threaded rods that adjust the angles of the first and second mirror assemblies to ensure that the image projected onto the display surface is aligned with the display surface. Includes rotatable adjustable nut 74c. In addition to orienting the projected image in the desired direction, the use of these two mirror assemblies allows longer focal length (and lower cost) projector lenses to be used with the projector. In addition, it provides a longer path between the projector 70 and the translucid layer 64a. In some alternative embodiments, an LCD panel or organic light emitting display (OLED) panel can be used in place of the video projector to display text and images, including a graphical user interface. Similarly, other techniques can be used to detect objects in contact with or in the immediate vicinity of the interactive display surface.

The above and subsequent discussions describe interactive display devices in the form of interactive display tables 60 and 60'. Still, it should be understood that the interactive display surface does not have to be in the form of a generally horizontal table top. The principles described in this description also adequately include and apply to display surfaces of different shapes and curvatures mounted in non-horizontal orientations. Therefore, the following discussion refers to placing a physical object "in contact" with the interactive display surface, but placing the physical object in contact with or otherwise close to the display surface. This allows the physical object to be placed close to the surface of the interactive display.

<u style="single">An exemplary way to implement an orientable GUI</u> Each of FIGS. 4 and 5 is a flow diagram illustrating steps of exemplary methods 400 and 500, respectively, to configure a user interface with dynamically orientable user input controls, as described in detail below. .. Methods 400 and 500 can be implemented in some embodiments using the components and techniques described above with reference to FIGS. 1-3. In some embodiments, one or more steps of methods 400 and 500 are performed on a computer-readable medium, including a computer-readable code, so that the computer-readable code is a PC. A series of steps will be performed when executed on a computing device such as a processor contained within 20. Subsequent discussions describe the various steps of methods 400 and 500 with respect to the processors of computing devices associated with interactive display systems that can perform certain method steps. However, the interactive display system can also (appropriately) communicate with another computing device that can also perform one method step. In some embodiments, certain steps of methods 400 and 500 can be combined, performed simultaneously, or performed in a different order without departing from the purpose of the method or producing different results.

Method 400 begins at step 410 when part of the graphical user interface (GUI) is selected. When that part is after the computer processor is allowed to execute computer executable instructions, such as those that can be included within an executable software module used to perform various functions. Can also be selected. In one exemplary embodiment, parts of the primary graphical interface can be selected when invoking an application that requires a dynamically orientable user interface for optimal functionality (eg, an interactive display). To allow more effective interaction with users located on different sides of the table). In one embodiment, the selected portion of the primary GUI defines a display field or display area in which a second portion of the GUI can reside. In some embodiments, the selected portion, in whole or in part, surrounds the main part of the GUI with a border. In other embodiments, the selected portion can be continuous around the periphery of the main portion of the GUI. However, the selected portion of the GUI can be any desired subset of the GUI so that the orientable GUI can be implemented within the selected portion. Therefore, the selected part does not need to be along the perimeter of the main part of the GUI, but can be placed closer to the center of the main part of the GUI and uses an orientable GUI. It can have any suitable shape that allows it.

At step 420, an orientable GUI is generated within the selected portion of the main part of the GUI. In some embodiments, the orientable GUI is independent of the main part of the GUI and can be dynamically oriented (eg, rotated) around the periphery of the main GUI. In another embodiment, the dynamically orientable GUI can be allowed to rotate freely within the selected part where the orientable GUI is generated. Provides a control unit. In yet another embodiment, the user control unit can be made rotatable along the axis of the selected portion. In some applications, the orientation of the user input control unit can be based on user input to the orientable GUI. As used herein, the term "orientable GUI" is oriented relative to a fixed main part of a GUI (eg, to be rotated or reoriented). A displayed image that can be moved in other ways). Thus, so that different specific users can more easily see and touch at least part of the orientable GUI, or at least part of the orientable GUI is around the interactive display table. The orientable GUI can be dynamically oriented so that it is displayed appropriately for easy reading and contact with different user positions. Specific examples of orientable GUIs are shown in Figures 6A-B and 7A-B, which are described below.

One embodiment of Method 400 includes additional steps, such as identifying when user input is received from a dynamically orientable GUI. In this embodiment, the current position of the user input control unit is first identified. The current position is determined relative to a given position within a selected portion of the display, such as the origin, which can be assigned when the orientable GUI is instantiated. In one example, the user input control unit can be, for example, a button that can be activated by the user with a finger or object positioned in the immediate vicinity of the button position on the surface of the interactive display. Another step of this embodiment determines whether the user input indicates a call to reorient or rotate the user input control unit relative to the current position of the control determined in the previous step. Can include doing. Based on this user input, the orientable GUI is dynamically drawn to rotate or reorient the user control within a selected portion of the interactive display where the orientable GUI resides. be able to.

In another embodiment, the step of identifying when user input is received by the orientable GUI is further close to the selected portion of the interactive display in which the orientable GUI is displayed. Includes steps to detect the placed object.

Another embodiment can include steps such as determining input parameters that correspond to physical phenomena of movement of the user's fingers or objects controlled by the user. In this embodiment, the parameters can include the speed and direction of user input. The determination can be based on user input directed to an orientable GUI, such as an input provided by a user's finger, hand, or an object held by the user. In the next step, the determined input parameters can be applied to dynamically control the speed and direction of change in the orientation of the user control section when the orientable GUI is drawn again. In general, the parameters associated with the physical phenomenon of motion are first calculated based on user input and then used during the drawing phase to control the display of the user control unit in the orientable GUI. However, in other embodiments, the entire orientable GUI field can be reoriented during drawing of the GUI on the display, based on the determined parameters according to the physical phenomenon of motion.

Another embodiment can include a step of determining a braking parameter. In some embodiments, the braking parameter can be proportional to the determined speed. In general, the braking parameter defines the rate of speed attenuation of reorientation (eg, rotation) of the user control section as the orientable GUI is being dynamically drawn. In some embodiments, the braking parameters can be based on user input received in the user interface. As an example, the braking parameters can be based on the detection of the user's hand or finger in the immediate vicinity of the area of the interactive display where the orientable GUI is displayed.

Another embodiment may include determining when a user input indicates a call to a user control unit and, in response, performing a predetermined function associated with that user control unit. Predetermined functions can include, for example, invoking an application, providing user input functionality to the application, and terminating the application.

Moving on to FIG. 5, a method 500 is shown for configuring a user interface for an interactive display with a GUI that can be dynamically oriented in response to user input. Method 500 is particularly suitable for implementation by a computing device and, in at least one embodiment, is implemented as a computer-readable instruction stored on a computer-readable medium. Method 500 is started in step 510 when the orientable GUI is selectively called. In some embodiments, the orientable GUI is independently controllable and dynamically orientable based on parameters such as user input. In general, an interactive display includes a main display field for user interaction that can display images and user interface elements. In some embodiments, the orientable GUI can be drawn in the sub-display field in the immediate vicinity around the primary display field. The orientable GUI can be selectively invoked whenever the interactive display application is active.

At step 520, an orientable GUI is generated in response to the call. In some embodiments, the orientable GUI is a user input control that can be activated by the user's finger, hand, or an object controlled by the user being brought into contact with or in close proximity to the control. Provides multiple user input fields such as departments. In other embodiments, the orientable GUI can be configured to allow dynamic modification of the orientable GUI along a path in a subfield that responds to user input to the orientable GUI. .. For illustration of this functionality, particular reference can be made to FIGS. 6A-B below. In yet another embodiment, the application selectively selects an orientable GUI to allow specific users to more easily access and interact with the controls or other parts of the orientable GUI. Can respond to redirection to.

Step 530 includes identifying when user input is received from the orientable GUI and analyzing that user input in response. Identification of user input can be done at any time after the orientable GUI is generated. In one embodiment, user input is received from an orientable GUI. In another embodiment, the user input is analyzed for input parameters such as the speed and / or direction of the object providing the input (or other parameters related to the physical phenomenon of movement).

At step 540, the orientable GUI is dynamically drawn based on one or more predetermined parameters. In one embodiment, when user input is received, the analyzed user input can also be used to dynamically control the orientation of the orientable GUI. In one embodiment, the orientable GUI is continuously and dynamically drawn at a rate sufficient to display an image moving to the interactive display surface without delay or other visual problems. Orientable GUI can be dynamically drawn on the interactive display many times per second during operation, and orientable GUI and user input control when the orientation of the orientable GUI and user input control unit changes. The appearance of the smooth movement of the part becomes possible.

Another embodiment involves identifying the current position of the orientable GUI relative to a given position on the interactive display. In this embodiment, the predetermined position is the anchor position, the currently identified position, or the initial position assigned when the orientable GUI is called, generated, last drawn, or first drawn. can do. This embodiment includes additional steps, such as determining whether the user input indicates a user call to displace the orientable GUI relative to the current position identified in the previous step. Can be done. Next, the position displacement (eg, reorientation) of the orientable GUI relative to the current position can be determined based on user input. The final step in this embodiment can include applying a determined position displacement to dynamically draw an orientable GUI in a new orientation.

Another embodiment involves determining input parameters for a physical phenomenon of motion based on user input received by an orientable GUI, such as input by a user's finger, hand, or object controlled by the user. Can include. Again, in this embodiment, the determined input parameters can be applied to dynamically control the speed and direction of displacement of the orientable GUI when the orientable GUI is drawn. It will be apparent to those skilled in the art that other parameters can be determined based on user input to the orientable GUI and that parameters can be used during the GUI drawing process. For example, another parameter that can be determined is the braking parameter. The braking parameter defines the rate of speed decay with respect to the displacement of the orientable GUI when it is rotating or otherwise moving while it is being dynamically drawn. be able to. Therefore, the braking parameter is the percentage of visible rotation of the orientable GUI over time as the orientable GUI finally reaches a complete "stop" state in the new orientation at the desired new displacement position. Slow down to. This braking parameter is based on a given model based on the laws of physics or user input such as the user's hand or finger when detected in the immediate vicinity of the area of the interactive display displaying the orientable GUI. Can be.

Another embodiment comprises detecting one or more nearby objects of a user input field in the orientable GUI when the orientable GUI is displayed on the surface of an interactive display. Can be done. This embodiment may include a further step of determining whether the user input indicates a call to the user control unit, and if so, performing certain functions associated with that user control unit. it can. In one embodiment, this predetermined function can include invoking a process independent of the orientable GUI control process, such as initiating the execution of a new application by an interactive control system. In another embodiment, this predetermined function can include providing a user control unit for an orientable GUI control process and an independent, currently running process.

An example of an orientable GUI control process and an independent process is a paint program that allows the user to "paint" on an interactive display surface with their fingers. In another embodiment, a given function can include providing a user control portion of an orientable GUI control process. In this embodiment, providing a user control unit to the orientable GUI control process itself is the appearance, rate of rotation, direction of rotation, or other orientation of the orientable GUI based on direct user input to the orientable GUI. It can include allowing control of parameters. In another embodiment, a given function can include terminating the process.

<u style="single">An exemplary orientable GUI displayed on the surface of an interactive display</u> 6A and 6B show one embodiment of the system and technique described with reference to FIGS. 1-5. Both FIGS. 6A and 6B show a top view of the interactive display surface 64, which is controlled to direct the GUI appropriate for the user 640 to see and interact with. Includes a dynamically orientable (eg, rotatable) GUI 610. As shown in FIG. 6A, the interactive display surface 64 has a peripheral portion 601 including side 602, side 603, side 604, and side 605. The apron 601 surrounds the interactive display surface as a whole, but is useful for supporting objects that can be used to interact with the user's hands and limbs as well as applications such as games run by the interactive display system. Although the interactive display surface 64 is illustrated as having four sides, the interactive display surface can actually have more or less than four sides, or be circular or elliptical. Please understand that you can. In this example, GUI The 610 is illustrated as a rectangular band extending around the periphery of the "active" display area of the interactive display surface 64. This "active" display area is the field of the interactive display surface 64 that contains both the orientable GUI 610 and the main part 620 of the user interface display that can be used to display other images. It should be noted that the GUI 610 can have any suitable shape and is not limited to bands along the periphery of the main part of the interactive display surface. The exemplary image 690 is represented by the word "image" that appears near the center of the main part 620 of the user interface display. The exemplary image 690 can be displayed, for example, as a separate user interface, as part of an application that runs independently of the GUI 610, as a picture, as a graph, or on an interactive display surface 64. Note that it can be almost any other type of image that can be.

In this example, the GUI 610 is typically such that the user controls within the fields of the GUI 610 move clockwise or counterclockwise along a line 650 that runs along the center of the bands that make up the GUI 610. , Main user interface configured to be oriented by "rotating" along the periphery of the display 620. Three user control units 621, 622, and 623, labeled "A," "B," and "C," respectively, are now placed in rows along side 604 and illustrated in GUI 610. Has been done. Three user control units 624, 625, and 626, labeled "1", "2", and "3", respectively, are now arranged in rows along side 603 and illustrated in GUI 610. Has been done. Further, a user control unit 630 (eg, a button) is illustrated as being called (or activated) by the user 640. In addition, the user's hand is also moving counterclockwise, rotating the GUI 610 counterclockwise at the same time as the user calls control 630.

During operation, the user control unit or button is called when the user's finger is placed on the user control unit in the position where it appears on the surface of the interactive display (ie, as shown in Figure 6A). .. In addition, the user's finger can then "drag" the called user control section along line 650 with the rest of the controls in the GUI 610. When the user's finger is removed from the interactive display surface 64, the user control unit dragged by the user's finger settles in a new position near the periphery of the main part 620 of the user interface display. The rotation of the user input button along line 650 can be stopped when the user's finger is removed, or it can continue after the user's finger is removed. GUI The continued rotation of the 610 is determined by a computing device placed in or coupled to the interactive display table, which is the finger as the user's finger moves over the surface of the interactive display. It can be based on the laws of physics, using parameters such as the measured speed and direction of. In this way, the user can "quickly move" the user input control with a sweeping motion at a particular speed (speed and direction), which allows the user to call the control and the GUI 610. It is possible to do both with initiating reorientation (or rotation).

FIG. 6B shows a further aspect of the example described above with respect to FIG. 6A, but the GUI 610 is shown in a displaced position or new orientation with respect to the position shown in FIG. 6A. The position or orientation of GUI 610 can be thought of as the current or initial position relative to the new position of this GUI after the user has reoriented this GUI. The displacements shown in Figure 6B are based on user input provided by sensing the movement of the user's fingers to demonstrate the rotational functionality of the GUI 610. As can be seen in FIG. 6B, the user input controls 621, 622, and 623 (ie, the buttons) have been rotated about 90 ° counterclockwise and are now positioned along the side surface 603. Similarly, user input controls 624, 625, and 626 (ie, buttons) have been rotated approximately 90 ° counterclockwise around the perimeter and are now positioned along side 602 of the interactive display surface 64. ing.

An exemplary embodiment is that the interactive display surface 64 has four sides and multiple users interact with the interactive display surface from their respective positions on different sides around the interactive display table. It can be useful if desired. In one example, the game can be played by four users accessing the game board displayed within the main user interface 620, and the different controls of the game are GUI Displayed along four sides of the interactive display 64 within the 610. Gaming applications can allow any user to access a particular set of controls more conveniently by reorienting the GUI so that the control activated by the user is positioned in front of that user. The user control unit can be dynamically oriented as needed. In this example, the player can "pass" control to nearby players playing from different sides of the interactive display 64, or is currently placed on a side of the interactive display table that is different from the side on which the player is placed. Allows a player who wishes to activate the desired control to rotate the desired control near the periphery of the interactive display surface until the desired control is placed in front of that player. It may be useful to provide rotational functionality to the user input controls within the GUI 610.

7A and 7B show another exemplary embodiment of the systems and techniques described with reference to FIGS. 1-5. 7A and 7B also show a schematic top view of the interactive display surface 64, which also displays an orientable user interface operated by the user. As shown in FIG. 7A, the interactive display surface 64 also has an apron 701, including side 702, side 703, side 704, and side 705. The orientable GUI 710 is also illustrated as having a band-like shape that extends near the periphery of the main display area 720. An exemplary image 790 is also represented by the word "image" located near the center of the main part 720 of the user interface display.

The orientable GUI 710 is generally rotatable near the periphery of the main user interface display 720 so that the fields of the orientable GUI 710 move in either the clockwise or counterclockwise direction. It is composed. Reorienting the GUI 710 in this way is similar to the functionality of a "rotary tray" tray. This is because the rotation of the rotating tray rotates an object placed in the rotating tray around the midpoint of the rotating tray for easy access by the user placed around the rotating tray. According to this analogy, the user input controls or buttons are similar to objects placed on the tray tray and are orientable GUI. The 710 is similar to a tray frame. This is because the orientable GUI supports a user input control section, which allows for dynamic reorientation of the GUI as it rotates around the main part 720 of the user interface display. Again, it should be noted that the dynamically orientable GUI does not have to be placed near the periphery of the main display area, but can be placed almost anywhere on the interactive display surface. Therefore, a dynamically orientable GUI can also be used, for example, in an application that can simulate the rotating wheel of a word game with this GUI. In such applications, a dynamically orientable GUI can be placed anywhere on the surface of the interactive display.

Three user control units 721, 722, and 723, labeled "A," "B," and "C," respectively, are now located in a contiguous line along side 703 within the GUI 710. Is illustrated in. A second set of user control units 624, 625, and 626, labeled "1", "2", and "3", respectively, are placed in consecutive rows along the side 705, GUI 710. Illustrated within. The user's finger 740 is shown touching a point in the GUI 710 along the side surface 704. In this embodiment, the user's finger is illustrated at a point in the GUI 710, but due to the fact that he has not touched and activated a user input control unit such as the user control unit 721, FIG. 6A and Distinguished from the embodiments shown in 6B. In Figure 7A, the user's finger is illustrated by rotating the orientable GUI 710 counterclockwise. As shown, the user's finger is placed on the GUI 710 and then the GUI "Drag" the 720 and all of its user input controls near the periphery of the main display area 720. When the user's finger is removed from the interactive display surface 64, the position of the GUI 710 settles in a new position near the periphery of the primary user interface display 720. The rotation of the GUI 710 can be stopped immediately when the user's finger is removed, or instead, such as speed and direction after the user's finger is removed, as described above. By applying the parameters, it is possible to keep rotating based on the laws of physics. Similarly, when the GUI 710 is moving, the user can stop the GUI 710 from rotating by simply placing his finger on a portion of the still-rotating orientable GUI.

In this way, the user inputs a part of the orientable GUI 710 by "flicking" it with a sweep to start the rotation of the orientable GUI 710. It is also possible to slow down the rotation over time at a rate of damping based on speed or at a predetermined rate of damping. FIG. 7B is a duplicate of FIG. 7A, except that the orientable GUI 710 is shown in a displaced position with respect to that shown in FIG. 7A. This displacement can be caused by user input (as shown in FIG. 7A) provided by the user's finger. As can be seen in Figure 7B, the user input controls 721, 722, and 723 have been moved approximately 180 ° counterclockwise near the periphery of the interactive display surface and are now positioned along the side surface 705. There is. Similarly, the user input controls 724, 725, and 726 have been moved approximately 180 ° counterclockwise near the periphery of the interactive display surface 64 and are now positioned along the side surface 703 of the interactive display surface. ..

However, it should be noted that the displacements shown in Figure 7B can be continuously variable, need not be limited to multiples of 90 °, and do not need to correspond to the sides of the interactive display 64. .. This ability to arbitrarily position the orientable GUI may be desirable if the user input controls are not specific to the user located on a particular side of the interactive display surface 64, or if the interactive display surface 64 is round. There is. In one example, a "paint" application could have multiple users around an interactive display surface 64, where users could select various "colors", sounds, and other effects. You must have easy access to the department. In this example, the user wants to access a particular desired application feature (eg, to select a different color range, etc.) regardless of the user's position relative to one "side" of the interactive display surface. ) You may want to rotate the control around the surface of the interactive display. In this way, an easily orientable GUI facilitates user access to the desired functionality.

Although various embodiments have been described with respect to preferred embodiments and modifications thereof for practicing what is considered novel, those skilled in the art will make a number of other modifications within the scope of the appended claims. You will understand what you can do with. Therefore, the scope of what is deemed novel by the present application is not limited in any way by the above description, but is determined solely by reference to the appended claims.

The new aspects in which the exclusive right is claimed are defined by the appended claims.

<figref num="1">A generally conventional computing device or personal computer (PC) suitable for use with an interactive display surface in practicing this technique of dynamically orienting a user interface presented on an interactive display surface. It is a functional block diagram which shows.</figref><figref num="2">FIG. 5 is a cross-sectional view showing an internal component of one exemplary embodiment having an interactive display surface at the top of an interactive display table that includes an integrated PC.</figref><figref num="3">FIG. 6 is an isometric view showing an exemplary embodiment in which an interactive display table is connected to an external PC.</figref><figref num="4">It is a flow diagram which shows the step of the exemplary method of constructing the user interface which has the user input control part which can be dynamically oriented.</figref><figref num="5">It is a flow diagram which shows the step of the exemplary method of constructing the user interface which has the user input control part which can be dynamically oriented in response to a user input.</figref><figref num="6A">FIG. 6 is a schematic top view showing an interactive display surface including a dynamically orientable user interface.</figref><figref num="6B">An interactive display surface that includes a dynamically orientable user interface that shows how the user interface shown in Figure 6A was reoriented for interaction by users on different sides of the interactive display surface. It is a schematic top view which shows.</figref><figref num="7A">FIG. 6 is a schematic top view showing an interactive display surface including a dynamically orientable user interface.</figref><figref num="7B">FIG. 7A is a schematic top view showing an interactive display surface showing a dynamically orientable user interface of FIG. 7A after the user interface has been reoriented.</figref>