Programmable on screen display and remote control

METHODS FOR PROGRAMMING A PROGRAMMABLE REMOTE CONTROL DEVICE AND A PROGRAMMABLE SCREEN DISPLAY BACKGROUND OF THE INVENTION

Field of Invention [001] The present invention relates, in general, to a programmable screen monitor and programmable remote control, through which the user can command operations of a controller or other programmable multimedia device.

Fundamentals of the Invention [002] Decades of long expansion in the number and types of consumer electronic devices have generated widespread use of radio frequency (RF) and infrared (IR) remote controls. Originally developed for television, remote controls now essentially accompany every CD player, DVD player, audio receiver, TV, home theater, camcorder, and portable radio. To simply play a DVD and watch a movie on a TV it may be necessary to operate three different remote controls in order to turn on and properly adjust the TV, DV player and home theater system for the presentation.

[003] A conventional approach to eliminate or reduce the number of physical remote controls is called universal remote control. In general, a universal remote control works by "learning" the RF or IR signals to which various devices respond. Once signals are learned and retained in memory, the universal remote can control multiple devices.

[004] Universal remotes, however, have certain disadvantages, starting with a stunning formation of physical buttons, perhaps dozens, which are often marked with mysterious, difficult-to-read labels. Due to the number of buttons, they are often so small that they are difficult to use and irritating. In addition, the physical layout

Petition 870190112194, of 11/04/2019, p. 11/37 / 14 of the buttons does not adapt to any industry standard, nor with an arrangement that most users find intuitive, or even familiar. Also, due to the small size of the button and counterintuitive layout, most users cannot operate a universal remote control by touching it in an “easy” way, but they often have to look down at the buttons to ensure that it is pressed. the correct. This represents a recurring, frustrating distraction for many users. Finally, a conventional universal remote control is unable to change its functionality based on different preferences, deficiencies or other user factors. SUMMARY OF THE INVENTION [005] In summary, the present invention provides both a programmable screen monitor and a programmable remote control, through which a user can command operations from a multimedia controller or other device. Through a graphical programming environment, the function of each programmable monitor button on the screen or physical button of that programmable remote control can be easily assigned or modified. Such a designated button function may vary by individual user to accommodate personal preferences, disabilities, parental controls, or other factors. Similarly, such a designated button function may vary according to a physical zone in which the user is present when using the button.

[006] According to one aspect of the invention, a designated button function is dynamically changed to provide appropriate functionality to the type of service that a user has selected. Similarly, a designated button functionality can dynamically change in response to a level or menu state that the user has navigated within a service. So, for example, when a user progressively accesses different services, the same button functionality can dynamically change from “channel up” (while controlling a cable TV box) to “initializing the call” (while controlling chat from video) in order to “obscure”

Petition 870190112194, of 11/04/2019, p. 12/37 / 14 (while controlling a lighting system) for any number of other desired features.

[007] The invention provides numerous advantages, including the elimination of the need for separate remote controls for various components that are interconnected with the multimedia controller. The present invention effectively enables a simple remote control, having a small number of large buttons, in a robust, powerful remote control that is intuitive and easily operated in an easy way.

[008] Another advantage of the invention is that it is able to effectively replace an expensive, bulky and non-portable touch screen. In addition, the programmable remote control, provided by the invention, is cheap enough that each member of a family can have one, each of which can be personalized in terms of functionality, access to services, preferences and the like. BRIEF DESCRIPTION OF THE DRAWINGS [009] The invention can be better understood by reference to the following description, together with the accompanying drawings, in which the same reference signs indicate identical or functionally similar elements:

Figs. 1A to 1D show a series of screen displays, in which a rotating selection of programmable buttons, in the form of icons representing different available services, is presented to a user who can use the buttons to control a programmable multimedia controller, according to an illustrative embodiment of the present invention;

Fig. 2 is a top plan view of a programmable remote control according to an illustrative embodiment of the invention;

Fig. 3 is a block diagram showing a high-level architecture of a system to dynamically assign functionality to the screen display buttons in Figs. 1A to 1D or to physical buttons on the programmable remote control of Fig. 2, according to an illustrative embodiment of the invention;

Petition 870190112194, of 11/04/2019, p. 13/37 / 14

Fig. 4 is a screen presentation showing how the system in Fig. 3 can be used to assign functionality to physical buttons on a programmable remote control for use with a cable TV service;

Fig. 5 is a screen presentation showing how that functionality assigned to a physical button on a programmable remote control can be dynamically changed in response to a user's action;

Fig. 6 is a screen presentation showing a graphical programming environment in which a collection of screen display menus is associated with a particular zone;

Fig. 7 is a screen presentation showing a graphical programming environment in which service requests to control a home lighting system are presented for assignment to screen display buttons or physical buttons on a programmable remote;

Fig. 8 is a screen presentation showing a graphical programming environment in which service requests to control cable TV are presented for assignment to screen display buttons or physical buttons on a programmable remote control; and

Fig. 9 is a screen presentation showing a graphical programming environment in which all possible commands to control a cable TV converter box are for assignment to screen display buttons or physical buttons on a programmable remote.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE PERFORMANCE [0010] Fig. 1A shows a screen display 2 for a programmable multimedia controller (not shown) or another device where a rotating selection of programmable buttons is presented to a user (not shown) ). Details regarding the structure, operation and services that can be performed by the programmable multimedia controller are set out in the related orders referenced above. In addition, for the sake of clarity,

Petition 870190112194, of 11/04/2019, p. 14/37 / 14 video images, graphics and other content are omitted from a screen display 2, however it must be understood that such content can be displayed on the screen together with the buttons on screen using, for example, the system and methods described in the copending application entitled “System and Method for Mixing Graphics with Video Images”, deposited concomitantly with it and assigned to a common assignee.

[0011] When pressed, button 4, which appears as an icon of an old-fashioned dinner music selector, will cause the programmable multimedia controller to start a “Play Media” service. Such a service can, for example, allow a user to play CDs, DVDs, MP3 files or other media. Button 6, which appears as a thermostat icon, when compressed, will cause the multimedia controller to start an HVAC service, through which the room temperature can be controlled. Button 8, which appears as a game controller icon, will cause the multimedia controller to start a video game service.

[0012] Other buttons, representing other available services, are shown in Fig. 1B. For example, a button 10, which appears as a TV icon, when compressed, will cause the multimedia controller to start a “Play Media” service. Such a service can, for example, allow a user to play CDs, DVDs, MP3 files or other media. Button 6, which appears as a thermostat icon, when compressed will cause the multimedia controller to start an HVAC service, through which the room temperature can be controlled. Button 8, which appears as a game controller icon, will cause the multimedia controller to start a video game service.

[0013] Other buttons representing other available services are shown in Fig. 1B. For example, a button 10, which appears as a TV icon, when pressed will cause the multimedia controller to start a cable TV service. Button 12, which appears as a

Petition 870190112194, of 11/04/2019, p. 15/37 / 14 a gadget, when pressed, will start displaying Mac-based gadgets on the screen. Button 14, which appears as a compact disc (CD) icon, when pressed, will start a CD playing service. It can also be seen in Fig. 1B that button 8 has turned three positions to the right from where it appears in Fig. 1A.

[0014] In a preferred embodiment, buttons 4 - 14, as well as others that may be present, rotate the display on screen 2 in response to a user action, such as pressing a button or applying pressure to a scroll wheel remote control, such as the one shown in Fig. 2 or performing some other action. When the user wants to “press” one of the tele buttons 4 - 14, he merely rotates the buttons on the screen until the desired one appears in the front and center position (p. eg button 4 in Fig. 1A or button 10 in Fig. 1B) on a screen display 2. At this point, the user can “press” the button on screen by pressing an appropriate button on a remote control, touching the corresponding area, if a touch screen is being used, or by any of numerous other actions.

[0015] Fig. 1C illustrates a screen display 16 that could appear, for example, in response to user pressure on the button on screen 10 (Fig. 1B) to start a cable TV service. That is, the screen display 16 can be considered, in a hierarchical sense, as a submenu or sublayer with respect to button 10 of a screen display 2. Three screen buttons 22, 24 and 26 are displayed, each receiving functionality that is appropriate for cable TV user control. Pressing button 22, for example, initiates access to “on demand” movies or other content available from a TV provider. Pressing button 24 starts a cable service provider “C” button and pressing button 26 initiates a cable TV box DVR “stop” command. As before, additional buttons (two of which are partially visible in reference number 20) can also be included in a rotating selection.

Petition 870190112194, of 11/04/2019, p. 16/37 / 14 [0016] Fig. 1D shows a screen display 18, which could appear, for example, in response to a user pressing a button to start a satellite TV service. Button 28, when pressed, initiates the cable service provider's “play” button. Button 30, when pressed, initiates button “A” from the cable service provider. Button 32, when pressed, initiates access to the cable service provider's “info” button. The 34 button initiates the cable provider's “exit” functionality, which is typically used when in a “guide” mode.

[0017] Fig. 2 shows a preferred embodiment of a programmable remote control 200, which can be used to rotate or press the screen display buttons of Figs. 1A-1B, as well as otherwise command the operations of a programmable multimedia controller or other device. In top plan view, remote control 200 includes a stand 202, five external buttons 204, 206, 208, 210 and 212 and four internal buttons 214, 216, 218 and 220. Remote control 200 also includes a central selection button 222. For simplicity, buttons 204 - 222 can be referred to individually or collectively here as "physical buttons".

[0018] In operation, the user using the remote control 200 can, in conjunction with a TV video monitor or other, command the operation of a multimedia controller or other device in an easy way, with the user looking generically at the monitor and not on the remote control 200. For example, by pressing button 208, a user can cause the on-screen buttons in Fig. 1A to rotate in one direction. By pressing in a circular motion around the external buttons 204 - 212, the user can make the buttons on the screen rotate in the opposite direction. By pressing button 222, the user can effectively “press” any of the buttons on screen 4 - 14 that is displayed in the front and center position of the screen, thereby initiating any service or action that is assigned to that button on screen.

Petition 870190112194, of 11/04/2019, p. 17/37 / 14 [0019] Fig. 3 is a block diagram showing a high-level architecture of a system for programming both the appearance and the functionality of the screen display buttons in Figs. 1A-1D, as well as the physical buttons on programmable remote 200 in Fig. 2. In a preferred embodiment where a Mac computer from Apple, Inc. is incorporated within a programmable multimedia controller, as described in the related orders referenced above, Adobe Photoshop® software, which is included with Mac OS X, is used to create the 300 files. That is, a Photoshop® 300 file is preferably created for each menu state of each button on screen, as well as each physical button.

[0020] In a preferred embodiment, each Photoshop file^® 300 contains information, preferably organized in “groups” and “layers”, which define the appearance, functionality and other attributes of each of the on-screen or physical buttons for a given menu state. Information for a given Photoshop file^® 300 includes notes, labels, text-to-speech commands, system control commands, graphics corresponding to different states of a button or other user control (eg enabled, pressed, over mouse), monitor text , font size, color, sound effect and button / control type, among others.

[0021] The 300 files, which are typically stored on a non-volatile medium, such as a hard drive, are read by Quartz Composer, another piece of software that is also included within Mac OS X, and a runtime environment that carries compositions from Quartz Composer. Quartz Composer has the function of analyzing Photoshop files^®300, thereby extracting all groups and layers embedded for the entire file and graphics, as well as for each button on screen or physical.

[0022] Quartz Composer “patches” access the extracted information and, using a Quartz Composer function called “exit ports”, pass appropriate commands and information from a patch to a

Petition 870190112194, of 11/04/2019, p. 18/37 / 14 access control application programming interface (API) 304 and user interface server (UI) 306. Details regarding the structure and operation of access control API 304 and UI 306 server are exposed in related orders referenced above.

[0023] This arrangement provides a powerful programming capability because it allows an author of a Quartz Composer composition to direct access to multiple layers of graphics, as well as any other information stored in the 300 files. In other words, simply by changing the information contained in a 300 file, an author, using widely available and familiar software, can easily and quickly configure, modify or customize any aspect of the appearance or functionality of a button on screen or physical button .

[0024] In response to commands and information received from patches via outbound ports, the access control API 304 and the UI server 306 issue commands and information appropriate to the service controller 308. The service controller 308, in turn, issues appropriate commands to external components 310 (eg, devices that are interconnected with a programmable multimedia controller) to perform whatever action a user has indicated. Details regarding the structure and operation of the 308 service controller are found in the related orders referenced above.

[0025] Fig. 4 is a screen image 400, representing how the system in Fig. 3 can be used to assign functionality to physical buttons on a programmable remote for use with a particular service, specifically a TV service cable. In the central part of screen image 400, an image 402 of a programmable remote control, like the one shown in Fig. 2, is displayed as a visual reference. In a 404 panel, the “layers” of a Photoshop® file are displayed. This file defines the functionality of each physical button on the programmable remote

Petition 870190112194, of 11/04/2019, p. 19/37 / 14 relation to a particular service, which in this example is simply running (watching) cable TV. As shown in panel 404, the physical button “navigate” 406 is assigned the function “ChannelAnalogUP” 408 which, within the environment of a programmable multimedia controller, operates to increment the channel in a cable TV converter box. Other attributes of the 406 button can also be specified by the Photoshop® file. For example, button 406 is identified as a button on a remote control 410 and is designated as a toggle type 412.

[0026] To enable a user to navigate through multiple levels or menu states, a desired button on the programmable remote control can be programmed with the “next menu” function. Similarly, a desired button can be programmed with the “retro menu” or “previous” function.

[0027] Fig. 5 is a screen image similar to Fig. 4, which illustrates how the functionality of the “navigate” button can be dynamically changed when the programmable remote control is used with respect to a different service state or menu . In Fig. 5, the physical “browse” button 506 is the same physical button as the “browse” button 406 in Fig. 4. However, in Fig. 5, a different Photoshop file^® is displayed on panel 504. This file defines the functionality of each physical button on the programmable remote with respect to playing FM radio. Here, the physical “browse” button 506 is assigned the function “Pre-adjustIncrement” 508 which, repeating, within the environment of a programmable multimedia controller, operates to increment to the next preset channel of an FM tuner, audio receiver or similar device .

[0028] The preceding description of how Photoshop® files can be used to assign and dynamically change the functionality of the physical buttons on a programmable remote control, is equally applicable to on-screen buttons like those shown in Figs. 1A-1D.

Petition 870190112194, of 11/04/2019, p. 20/37 / 14 [0029] In the related order PROGRAMMABLE MULTIMEDIA CONTROLLER WITH PROGRAMMABLE SERVICES there is a detailed description referring to “user profiles” and “zones”. In general, user profiles are files or other data that include information about an individual user's preferences, access rights or restrictions or other information. In general, zones are particular spaces or physical areas in which particular devices are present, which are interconnected with and under the control of a programmable multimedia controller. In conjunction with the graphic programming environment described in the order related to the title PROGRAMMING ENVIRONMENT AND METADATA MANAGEMENT FOR PROGRAMMABLE MULTIMEDIA CONTROLLER, the present invention can be advantageously used to provide on-screen or physical button functionality, which can be dynamically changed according to user profile or zone, where a user is located, or both.

[0030] Fig. 6 is a screen image of a 600 programming environment. A graphical representation of a zone called “Lab Station 1” is represented on a panel 602. A partial graphical representation of a second zone called “Lab Station 2 ”is represented on a panel 604. A panel 606 is a graphical representation of screen display menus, which has been programmed to appear in the Lab Station 1 zone. As indicated, in general, by connector lines 608, the screen display menus of panel 606 are associated with appropriate control ports, inputs or outputs, which are present in the Lab Station 1 zone. Similarly, the transitions between display menus can be graphically programmed as desired.

[0031] As shown in panel 606, a user called “Standard User” 610 is identified and is herarchically associated with a user profile called “Standard Profile” 612. At the next level of the hierarchy, the Lab Station 1 zone is associated with the user profile 612 and, at the next level, the screen display menus 614 are associated with the Lab Station 1 zone.

Petition 870190112194, of 11/04/2019, p. 21/37 / 14 next lower level, although not visible in this figure, are the individual buttons associated with the on-screen menu 614. Additional user profiles can be associated with a given zone using a mouse to select the user row and then clicking on the “+” button located on the bottom left of panel 606.

[0032] The ability to associate user profiles with screen display menus is highly advantageous in many ways. For example, a user who is left-handed may prefer that the functionality assigned to buttons on a programmable remote control be inverted, compared to a user who is right-handed. By specifying this preference in the user profile and dynamically adjusting the functionality assigned to on-screen and physical buttons, the present invention effectively personalizes and optimizes the user experience. In another example, if the user is a child, by specifying this information in the user profile, the present invention allows the exercise of source control to prevent certain screen display menus from appearing, blocking access to certain means and the like.

[0033] Fig. 7 is a screen image of a graphical programming environment 700, in which user actions, also referred to as “service requests”, of the related requests referenced above, to control such a domestic lighting system ( not shown), are presented to assign to desired screen display buttons or physical buttons on a programmable remote. In a 702 dialog box, a left part is a list of 704 services, which can be provided by a programmable multimedia controller that is interconnected with appropriate external components. The first service listed is “Lighting Control Service”, which is shaded to indicate that 706 service requests, listed in the right part of dialog 702, correspond to “Lighting control service”. As shown, the available 706 service requests are called ButtonPress, ButtonPressAndRelease, ButtonRelease, DimmerSet,

Petition 870190112194, of 11/04/2019, p. 22/37 / 14

SwitchOff and SwitchOn. In other words, those available service requests 706 represent actions that a user can take in relation to the “Lighting Control Service”, which will be recognized by the programmable multimedia controller as valid actions.

[0034] Any of those 706 service requests can be assigned to any of the screen display buttons or physical buttons on a programmable remote control, which appear on panel 606, which is the same panel shown in Fig. 6, except that the screen display menu called “RotateLighting” is now expanded and displayed. To assign a particular service request to a particular button, simply use a mouse to drag the service request 706 and release it on the desired 606 button.

[0035] Fig. 8 shows a graphical programming environment 800, in which service requests 806, to control a cable TV service 804, are presented for assignment to the desired on-screen or physical buttons. As described above, any service request 806 can be assigned to any button present on panel 606 by drag and drop. A screen display menu called RotateCableTv is expanded and displayed on panel 606.

[0036] Fig. 9 shows a graphical programming environment 900 representing an alternative method for assigning a user command to a screen display button or physical button. Panel 902 presents a list of all possible 902 user commands to control a cable TV converter box. Similar lists (not shown) can be provided for any desired device group (e.g. eg receivers, DVD players, TVs etc.) to include a master collection of all devices for which the necessary information can be collected. With panel 902 displayed, any of the commands listed can be assigned to any screen display button or physical button on panel 606 by drag and drop.

Petition 870190112194, of 11/04/2019, p. 23/37 / 14 [0037] The foregoing description has been directed to particular embodiments of this invention. It will be evident, however, that other variations and modifications can be made in the described embodiments with such an achievement of some or all of its advantages. In addition, the procedures or processes could be implemented in hardware, software, incorporated as a computer-readable medium with program instructions, firmware or a combination of them. Therefore, it is an objective of the appended claims to cover all such variations and modifications that fall within the true scope of the invention.