GUI transitions on wearable electronic device

GUI transformation technology field on wearable electronic devices

[0001] This application requires US provisional patent application No. 61/728765 filed on November 20, 2012, and US provisional patent application No. 61/728770 filed on November 20, 2012, in accordance with 35 USC § 119(e). U.S. Provisional Patent Application No. 61/773803 filed on March 6, U.S. Provisional Patent Application No. 61/728773 filed on November 20, 2012, U.S. Provisional Patent Application No. 61/773813 filed on March 7, 2013, 2013 U.S. Provisional Patent Application No. 61/773815 filed on March 7, 2013, U.S. Provisional Patent Application No. 61/773817 filed on March 7, 2013. U.S. Provisional Patent Application No. 61/775688 filed on March 11, 2013, U.S. Provisional Patent Application No. 61/775687 filed on March 11, 2013, U.S. Provisional Patent Application No. 61/775686 filed on March 11, 2013, and U.S. Application No. 14/015,890 filed on August 30, 2013 , All of these applications are incorporated herein by reference.

[0002] The present disclosure generally relates to wearable electronic devices.

Background technique

[0003] Mobile electronic devices provide users with access to computing power, even when the user moves around in various locations. Examples of mobile electronic devices include mobile phones, media players, laptop computers, tablet devices, PDAs, or hybrid devices that include the functionality of multiple such devices.

[0004] The mobile electronic device may be part of a communication network such as a local area network, a wide area network, a cellular network, the Internet, or any other suitable network. Mobile electronic devices can use communication networks to communicate with other electronic devices, for example, to access remotely stored data, access remote processing capabilities, access remote displays, provide locally stored data, provide local processing capabilities, or provide access to local displays. For example, a network can provide communication paths and links to a server, and the server can host applications, content, and services that can be accessed or utilized by users via mobile electronic devices. The content may include text, video data, audio data, user settings, or other types of data. The network can use any appropriate communication protocol or technology to facilitate communication between mobile electronic devices, such as Bluetooth, IEEE WI-FI (802. lla/b/g/n/ac) or TCP/IPo.

[0005] Solution to Problem

[0006] In one embodiment, an apparatus includes one or more processors, and a memory coupled to the processor including instructions executable by the processor. When the instructions are executed, the processor presents the first screen of the graphical user interface on the display of the device. The first screen includes one or more first elements. The processor receives user input indicating a transition in the graphical user interface, and in response to the user input, transitions from the first screen to the second screen of the graphical user interface and applies one or more visual transition effects to the transition. The second screen includes one or more second elements.

[0007] Advantageous effects of the invention

[0008] According to the present invention, a method and device for GUI transition on a wearable electronic device are provided.

Description of the drawings

[0009] FIG. 1 illustrates an example embodiment of a wearable electronic device.

[0010] FIG. 2 illustrates an example stack of devices.

[0011] Figures 3A-3E illustrate example form factors of the device.

[0012] FIG. 4A illustrates an example cross-section of the device body.

[0013] FIGS. 4B-4C illustrate example connections between components of the device.

[0014] Figures 5A-5F illustrate example displays of the device.

[00 Figures 6A-6C illustrate example cross-sectional views of the device display.

[0016] FIGS. 7A-7D illustrate example external elements surrounding the body of the device.

[0017] FIGS. 8A-8C illustrate example external elements surrounding the body of the device.

[0018] FIG. 9 illustrates an example seal ring of the device.

[0019] FIG. 10 illustrates an example retention ring of the device _o

[0020] FIG. 11 illustrates various example embodiments of a wearable device.

[0021] FIGS. 12A-12B illustrate a strap attached to the main body of the device.

[0022] FIGS. 13A-13I illustrate an example embodiment of a strap for fastening or fixing the device.

[0023] Figures 14A-14D illustrate example camera placement on the device.

[0024] FIG. 15 illustrates an example device with a strap and an optical sensor.

[0025] FIG. 16 illustrates an example viewing triangle including users, devices, and objects.

[0026] FIG. 17 illustrates an example viewing angle of the optical sensor of the device.

[0027] Figures 18A-18B illustrate an example optical sensor of the device.

[0028] FIG. 19 illustrates an example sensor detection system of the device.

[0029] FIGS. 20A-20C illustrate an example charger that can operate in conjunction with a device.

[0030] FIGS. 21A-21B illustrate an example charger that can operate in conjunction with a device.

[0031] FIGS. 22A-22B illustrate an example charging unit that can be operated in conjunction with a device.

[0032] FIG. 23 illustrates an example charging scheme of a charging unit operable in conjunction with a device.

[0033] FIG. 24 illustrates an example charging scheme of a charging unit operable in conjunction with a device.

[0034] FIGS. 25A-25E illustrate example embodiments of energy storage and charging and charging units in a device

[0035] FIG. 26 illustrates an example charging unit architecture.

[0036] FIGS. 27-92 illustrate example gestures for the device.

[0037] FIGS. 93A-93B illustrate example user input to the device.

[0038] FIGS. 94A-94C illustrate example user input to the device.

[0039] FIGS. 95A-95D illustrate example user touch input to the device.

[0040] FIGS. 96A-96B illustrate an example graphical user interface model of the device.

[0041] FIG. 97 illustrates an example graphical user interface model of the device.

[0042] FIGS. 98A-98G illustrate example graphical user interface models of the device.

[0043] FIG. 99 illustrates an example graphical user interface model of the device.

[0044] FIGS. 100A-100C illustrate example graphical user interface models of the device.

[0045] FIGS. 101A-101B illustrate example screens of the graphical user interface of the device.

[0046] FIGS. 102A-102D illustrate example screens of the graphical user interface of the device.

[0047] FIGS. 103A-103D illustrate example screens of the graphical user interface of the device.

[0048] FIG. 104 illustrates an example menu of the graphical user interface of the device.

[0049] FIGS. 105A-105D illustrate example menus of the graphical user interface of the device.

[0050] FIGS. 106A-106C illustrate example menus of the graphical user interface of the device.

[0051] FIGS. 107A-107C illustrate example menus of the graphical user interface of the device.

[0052] FIG. 108 illustrates an example menu of the graphical user interface of the device.

[0053] FIGS. 109A-109C illustrate example menus of the graphical user interface of the device.

[0054] FIGS. 110A-110B illustrate examples of scrolling in the graphical user interface of the device.

[0055] FIGS. 111A-111C illustrate examples of scrolling in the graphical user interface of the device.

[0056] FIG. 112 illustrates an example of overlay and background content in the graphical user interface of the device.

[0057] FIGS. 113A-113C illustrate examples of overlay and background content in the graphical user interface of the device.

[0058] FIGS. 114A-114B illustrate example visual transition effects in the graphical user interface of the device.

[0059] FIGS. 115A-115B illustrate example visual transition effects in the graphical user interface of the device.

[0060] FIGS. 116A-116B illustrate example visual transition effects in the graphical user interface of the device.

[0061] FIGS. 117A-117B illustrate example visual transition effects in the graphical user interface of the device.

[0062] FIGS. 118A-118C illustrate example visual transition effects in the graphical user interface of the device.

[0063] FIGS. 119A-119C illustrate example visual transition effects in the graphical user interface of the device.

[0064] FIGS. 120A-120C illustrate example visual transition effects in the graphical user interface of the device.

[0065] FIGS. 121A-121B illustrate example visual transition effects in the graphical user interface of the device.

[0066] FIG. 122 illustrates an example use of the physical model in the graphical user interface of the device.

[0067] FIG. 123 illustrates an example screen of the graphical user interface of the device.

[0068] FIG. 124 illustrates an example screen of the graphical user interface of the device.

[0069] FIG. 125 illustrates an example method of automatic camera activation in a device.

[0070] FIG. 126 illustrates an example method of delegation by a device.

[0071] FIG. 127 illustrates an example delegation model including equipment.

[0072] FIG. 128 illustrates an example method of delegation by a device.

[0073] FIGS. 129A-129D illustrate example modes of the device.

[0074] FIG. 130 illustrates an example mode of a device.

[0075] FIGS. 131A-131D illustrate example modes of the device.

[0076] FIG. 132 illustrates an example method of providing augmented reality functionality on a device.

[0077] FIG. 133 illustrates an example network environment in which a device can operate.

[0078] FIG. 134 illustrates an example of pairing between a device and a target device.

[0079] FIG. 135 illustrates an example method of pairing a device with a target device.

[0080] FIG. 136 illustrates an example screen of the graphical user interface of the device.

[0081] FIG. 137 illustrates an example computer system including a device.

Detailed ways

[0082] In one embodiment, an apparatus includes one or more processors, and a memory coupled to the processor including instructions executable by the processor. When the instructions are executed, the processor presents the first screen of the graphical user interface on the display of the device. The first screen includes one or more first elements. The processor receives an indication of a transition in the graphical user interface

In response to the user input, transition from the first screen to the second screen of the graphical user interface and apply one or more visual transition effects to the transition. The second screen includes one or more second elements.

Detailed ways

[0083]

[0084] FIG. 1 illustrates an example embodiment of a wearable electronic device 100. The device 100 includes a main body 105, and the main body 105 contains all or some of the circuits, structures, and displays of the device 100. For example, the main body 105 may include all or some of the processing components, data storage components, memory, sensors, wiring, or communication components of the device 100. In certain embodiments, the device 100 may include a display. The display can take any suitable form or shape, such as a circular shape, as shown in a circular display 110. As used herein, "circular display" includes a substantially circular display or a display resembling a circle, such as an elliptical display, as appropriate. In certain embodiments, the device 100 may include elements surrounding the display. As used herein The element surrounding the display includes a rotatable element surrounding the display or a body in or on the display. As an example, the element may be an outer ring 115 surrounding the circular display 110. In a particular embodiment, the element surrounding the display may Move relative to the display or the main body. For example, the outer ring 115 can rotate relative to the main body of the device 100, which is described more fully below. In certain embodiments, the device 100 may include a strap 120 attached to the main body 105. In certain implementations In an example, the device 100 may include a device fixed in the main body 105 or the strap 125 or fixed to the main body 105 or the strap 125 The sensor module, such as the camera module 125 that houses the camera, is described more fully below.

[0085] A specific embodiment of a wearable electronic device includes a stack that allows some or all of the processing and display system to be placed inside the body of the device, and the body of the device can be provided with at least one such as an outer ring for the user Surrounded by elements of the way the device interacts. As an addition or alternative, certain embodiments may include external components incorporated into the strap for additional functions, which will be more fully described herein. Figure 2 illustrates an example stack 200 of wearable electronic devices. As shown in FIG. 2, some or all of the components of the stack 200 may take the form of a device, which in the example of FIG. 2 is circular. The stack 200 may include a layer of protective glass (or other suitable transparent solid material) 205. Other components may be laminated to the protective glass 205 or attached to the substrate 245. As an addition or alternative, the protective layer 205 may be mechanically connected to the outer ring 235, or any other suitable component of the main body of the device. Directly below the protective glass 205 may be the touch sensitive layer 210. The touch sensitive layer 210 may be composed of any suitable material and may be of any suitable type, such as resistance, surface acoustic wave, capacitance (including mutual capacitance or self capacitance), infrared, optical, dispersion, or any other suitable type. The touch sensitive layer 210 can be directly applied to the protective glass 205, laminated to the protective glass 205, or physically fixed to the Protect glass 205. The touch-sensitive layer 210 may be a completely two-dimensional touch surface, or may be composed of touch-sensitive areas, such as several capacitive buttons or areas. The touch sensitive layer 210 may be connected to the processor board 215 via a flexible connector at the edge of the touch surface, which will be more fully described herein.

[0086] Below the touch sensitive layer 210 may be a circular display 215, which may be laminated or mechanically fixed to any previous or aforementioned layer. In certain embodiments, lamination can reduce glare and improve display legibility by reducing internal reflection. As described more fully below, the display 215 may have an external inactive area, which may be symmetrical or asymmetrical. The display 215 may be positioned such that it is axially centered relative to the protective layer 205 in order to obtain a visually symmetrical presentation. The display 215 may be of any suitable type, such as a light-emitting diode (LED), an organic light emitting diode (OLED), or a liquid crystal display (LCD)._oIn certain embodiments, the display 215 may be flexible. In certain embodiments, the display 215 may be partially transparent. In certain embodiments, the display 215 may be translucent.

[0087] Below the display 215 may be a battery 220, which in certain embodiments may be positioned such that the diameter of the base 245 may be reduced without affecting the size of the battery. The battery 220 may be of any suitable type, for example, based on lithium ion. The battery 220 may adopt the circular shape of the device, or may adopt any other suitable shape, such as a rectangular shape, as shown in the figure. In certain embodiments, the battery 220 may be "floated" in the device, for example, there may be space above, below, or around the battery to accommodate thermal expansion. In certain embodiments, for optimal packing of the components, higher-height components such as haptic actuators or other electronic devices may be positioned in the extra space beyond the edge of the battery. In certain embodiments, the connector from the processor board 225 can be placed in this space to reduce the overall height of the device.

[0088] Below the battery 220 may be the processor board 225. The processor board 225 may include any suitable processing components, such as one or more processing units, driving units, sensing units, caches, memory elements, or integrated circuits. The processor board 225 may include one or more thermal sensors or cooling units (e.g., fans) for monitoring and controlling the temperature of one or more processor board components. In certain embodiments, the body 105 of the device itself can act as a heat sink.

[0089] Below the processor board may be an encoder 230, which is surrounded by one or more outer rings 235. As described more fully below, the encoder 230 may be of any suitable type, and may be part of the outer ring 235 or may be a separate component, as shown in FIG. 2. In a specific embodiment, the outer ring 235 may provide detent tactile sensing of the outer ring or position sensing of the outer ring 235. When the encoder 230 is a mechanical encoder separate from the main body of the device, as shown in FIG. 2, the encoder may support the outer ring 235. For example, in certain embodiments, the encoder 230 is mounted to the base 245, and the connection to the base 245 or to the strap 240 may pass through some part of the encoder, such as the center of the encoder. In certain embodiments, the processor board 225 and one or more layers above may be attached to the center post passing through the encoder 230. The column can transfer mechanical forces on the components of the device to the column, which can allow components such as the processor board and display to be supported by the column instead of the encoder, which reduces strain on the encoder. In a particular embodiment, the outer ring 235 is attached to the movable part of the encoder via a prong or other suitable connection.

[0090] The main body of the device may end at the base 245. The base 245 may be stationary relative to one or more rotatable components of the device-such as the outer ring 235. In certain embodiments, the base 245 is connected to a strap 240 that will be described more fully herein. The connection may be mechanical or electrical, for example, a part of the circuit links the wired communication component in the strap 240 to the processing board 225. In certain embodiments, the connector is positioned to avoid the anchor point of the encoder and strap. In certain embodiments, the strap 240 can be detached from the base 245. As will be described more fully herein, the strap 240 may include one or more internal connectors 250, one or more optical sensing modules 255, or one or more other sensors. In certain embodiments, the inside of the device or parts of the inside may be sealed with respect to the external environment.

[0091] Although the present disclosure describes the components in the stack 200 of the wearable electronic device 100 and specific examples of the shape, size, order, connection, and function of these components, the present disclosure contemplates that a wearable device such as the device 100 can be This includes any suitable components of any suitable shape, size and order connected or communicated in any suitable manner. As just one example, the battery 220 may be placed closer to the bottom of the stack than shown in FIG. 2. As another example, the main body of the device may take any suitable form factor, such as an ellipse or a disc shape as shown in the example of FIG. 3A, one end tapering as shown in the example of FIG. The edge 315 shown in the example of FIGS. 3C-3D is chamfered or rounded at one or more edges. Figure 3E illustrates additional example form factors of the device body, such as a body 320A-Eo with a polygonal shape with a flat protective cover or display or a curved protective cover or display. As another example, the main body 325A-D has a flat protection Part of the curved shape of the mask or display or bend protection mask or display. The main body 330A-C has a curved shape. One or more internal components of the main body of the device, for example, one or more internal components, can take any form factor suitable for the main body in which it is located.

[0092] FIG. 4A illustrates an example cross-section of the device body. As shown in the figure, the device body has a width of D1, for example, about 43 mm. Certain embodiments may include a small gap D4 between the outer ring and the OLED display, such as a gap of up to 0.3 mm. Similarly, in the outer ring and the glass protective shield (which may have a width D3, for example, about 42. There can also be a distance between 6 mm), for example · 2 mm. In a specific embodiment, the gap between the protective glass cover and the outer ring is larger than the gap between the display and the outer ring. The outer ring (which may include serrations) may have a width D2 of, for example, 1.0 mm. Figures 4B-4C illustrate example connection groups between the components of the device. FIG. 4B illustrates the touch glass 405 above the display 410. The display is attached to the top of the inner body 440 with, for example, an adhesive sealant 425. The display flexible printed circuit 430 couples the display to the electronic device in the main body of the device. The adhesive sealing film 445 can be used to connect the strap 450 to the device, and one or more retention rings 435 can be used to connect the outer ring 415 to the inner body 440. In certain embodiments, the retention ring can inhibit the outer ring from twisting on its vertical axis and provide a physical distance between the outer ring and the glass cover. A layer of protective glass can be located on top of the inner body to provide environmental sealing. In certain embodiments, the retention ring may also provide an environmental seal for the inner body. For example, Figure 5C illustrates an example retention ring 465 attaching the outer ring to the device body and providing an environmental seal between the outer ring and the inner body. As an addition or alternative, a flock-type material that may be coated with a hydrophobic substance such as TEFLON can be used to prevent water and dust from intruding into the cavity. As another example, a metal or plastic ring can be used to close the outer ring. Sealed to the inner body to prevent air (and thus water vapor or other particles) from moving through the cavity between the outer ring and the inner body. The gap 455 allows the outer ring to move relative to the inner device body, for example by rotating it. The adhesive sealant 460 attaches the display to the main body and provides an environmental seal between the display and the components of the inner main body.

[0093] In a specific embodiment, the display of the device has a circular or elliptical shape, and houses a circular display unit, such as an LCD display, and an OLED display. The display unit can be placed such that the visible area is centered in the display module. If the display unit has an offset design, one or more suitable masks can be used to mask a portion of the display to create a circular and correctly placed visual outline.

[0094] In a particular embodiment, the display module has an outer ring that is part of the user interface of the device. While the strap keeps the bottom and the inside of the device stable, the outer ring can rotate. Figure 5A illustrates an example of a top view of the display of the device relative to other device components. The outer ring 510 may be attached to the front surface 512 of the device 508, or it may be independent of the front surface 512. In a particular embodiment, no matter how the outer ring 510 around the display 506 rotates, the display 506 does not rotate. This can be achieved by attaching the display 506 to the portion 504 of the display module that is fixed to the strap 502, or by programming the displayed content to remain stationary while the display unit rotates. In the latter case, the displayed content is rotated so that the visual vertical axis of the image displayed by the display unit always remains parallel to the strap.

[0095] The display module may additionally include one or more sensors on the same surface as the display or in the vicinity thereof. For example, the display module may include a camera or other optical sensors, a microphone, or an antenna. One or more sensors can be placed in the inactive area of the display. For example, FIG. 5B illustrates the device 522 in which the camera module 516 is placed under the display 520 coplanar with the battery, and the optical opening 514 is positioned under the clear portion of the display 520. The camera module 516 may be placed between the grid line connectors 518 of the display 520. Any camera or other suitable sensor may be placed coplanar with the display, such as the antenna 524 of FIG. 5C, which is placed in the inactive area 526. As an addition or alternative, the sensor can be placed below or above the display, can be placed in any suitable position in or on the outer body of the device, and can be placed in or on the strap of the device. Location, or any appropriate combination of the above, which will be more fully described herein. For example, a front-pointing camera can be placed below, on, or above the display.

[0096] In a particular embodiment, the package of the circular display includes an inactive area, as shown in FIG. 5D. In traditional displays, the row drive lines that power the display are directed to the nearest side edge, and then or down the inactive area

Lead the way, or connect directly to the driver integrated chip along the edge. Several solutions can be taken to reduce the amount of inactive areas of the display. For example, certain embodiments reduce the size of the inactive area by redirecting the grid control line that powers the display to one edge of the display. FIG. 5D illustrates grid control lines 532 routed to one edge of the display 536 and connected to the connector 538, which leads these lines to the processing center of the device 528. In this configuration, the inactive area 530 can be minimized.

[0097] FIG. 5E illustrates a further example embodiment for reducing the inactive area of the display 554 of the device 540 by creating a polygon-type display profile, the display profile having a display profile that is masked by one or more at the center (Mask) 550 masked circular area. The connector 552 is arranged in a polygonal design. Row 546 and column 542 of the grid line are routed to the nearest connector 552. In a specific embodiment, the connector 552 is connected to a flexible circuit carrying a driver chip behind the display. Due to the reduction in connection density, the electronic device of FIG. 5E can be more easily connected to a flexible printed circuit board (FPC board) and thereby increase yield. In addition, by moving the driver integrated circuit to the back of the display, one or more inactive regions 548 can be further reduced while allowing the integrated circuit to remain on a stable and flat surface. This design is particularly suitable for OLED displays, but can also be used for LCDs, considering that the backlight unit (BLU) can be laminated to the device before connecting the FPC board. Although the above example illustrates a polygonal arrangement of connectors, any suitable arrangement of connectors can be used as long as the grid lines reach all pixels.

[0098] FIG. 5F illustrates an example physical arrangement and size setting of the display of the device. The device has a diameter of D4, for example about 41.1 mm. The device includes one or more inactive areas having a width D3, for example about 1.55 mm. The device includes a visible area with a diameter D2, for example about 38 mm. The device includes connectors 568 for column lines 564 and row lines 566. The connector 568 may be coupled to the device by one or more FPC bonds 570, these FPC bonds 570 having a width of D1, for example, about 0.2 mm. The connector 568 may have a width D5, for example, about 6 mm. The display connector FPC 556 can be used to connect electronic devices of the display, such as the circuit from the connector 568, to the driver chip 558, which can be under the display or behind the main body of the device.

[0099] FIGS. 6A-6C illustrate example cross-sectional views of a device display, including the manufacture of the device. In Figure 6A, a hotbar knife 605 is used to solder the flexible printed circuit(s) 610 that couples the electronics of the display to the processing electronics of the device. The support 615 can be used to stabilize the FPC 610 during this process. Figure 6B illustrates the connected FPC 620, which is folded up (part 625) and glued to the back of the display with an adhesive 630. Figure 6C illustrates an example completed display. The FPC 645 is laminated to the back of the protective display glass 635, and is bent to the front surface of the glass 635 and attached to the front surface of the glass 635 via micro bonding 649. The adhesive 650 connects the FPC 645 to the device. The FPC crosses the driver chip 655, which is connected to the device by an adhesive 650.

[0100] In certain embodiments, all processing and RF components are located within the body of the device, which can create challenges in allowing RF signals to pass out of the device. The FPC board can be additionally attached to the side of the polygon that does not go to the connection to the display itself to allow the installation of a wire, stub, ceramic or other antenna (or other appropriate sensor) in the same plane as the display, as shown in the figure As shown in 5C. Since the antenna of FIG. 5C is coplanar with the display, interference from the dense grid of the wiring from the display (such as shown in FIG. 5E) is reduced.

[0101] In certain embodiments, a metal shield can be used to shield the display from electromagnetic interference with the main processor board. In certain embodiments, the metal shield can also be used as a heat sink for the battery, thereby increasing the charging or discharging rate of the battery.

[0102] In certain embodiments, the wearable electronic device may include one or more external elements (they may be any suitable shape) surrounding the device body. FIG. 7A illustrates external elements with an example outer ring 710 surrounding the display 705. As shown in FIG. The outer ring can be constructed of any suitable material, such as stainless steel or aluminum. In certain embodiments, the outer ring 710 may be in a

Rotate in one direction, rotate in two directions, or can be used based on, for example, a switch in these two configurations. In certain embodiments, one outer ring 710 can rotate in one direction, and the second outer ring 710 can rotate in the opposite direction. The outer ring 710 may be coupled to the base 720 of the device by a retention ring 715. Figure 7B illustrates the outer ring 710 attached to the base 720 by a Delrin ring 715A or a spring steel retention ring 715B. A spring or clip 725 secures the ring to the base 720. 7C-7D illustrate the retention ring 715 secured to the base 720 via screws 725 screwed into the corresponding posts of the base 720. FIG. The device may include fasteners/spacers 730, as shown in Figure 7C.

[0103] In certain embodiments, a stopper or an encoder of an external element (the two can be used interchangeably when appropriate) can provide the user with tactile feedback (such as a tactile click), such as Provided by a stopper that allows the user to determine when the element has been moved a "step" or an "increment" (the two are used interchangeably in this document). This click sensation can be directly generated via mechanical linkage (such as a spring mechanism), or it can be generated electronically via a tactile actuator (such as a motor or a piezoelectric actuator). For example, the motor may provide resistance to the movement of the ring, such as by being short-circuited to provide resistance and un-short-circuited to provide less resistance, thereby simulating the relatively high and low torques provided by the mechanical stop system. As another example, a magnetic system can be used to provide the tactile feel of the stopper. For example, a solenoid mechanism can be used to release the stop spring or escapement device as needed. A spring or escapement device provides actual mechanical feedback. However, this arrangement allows the device to skip several stoppers as needed, while re-engaging the stoppers at precise intervals to produce the feeling of a stopper. For example, the one with a changed size. As another example, a rotatable external element (such as an outer ring) may be magnetized, for example by an electromagnetic used to attract the ring in a "stop" position, thereby increasing the torque and simulating a stop feedback. As another example, the rotatable external element may have alternating north-south poles, which repel and attract corresponding magnetic poles in the body of the device. As another example, permanent magnets can be used to lock the ring in place when the electromagnet is not in use to prevent coasting. As another example, instead of electromagnets, ferromagnetic alloys that are easily magnetized can be used in solenoids. This allows the electromagnetic field of the solenoid to "reprogram" the magnetic orientation of the magnetic core, thereby maintaining the effect of magnetic actuation even when the solenoid itself is released. Although the present disclosure provides specific examples of stops, stop-like systems, and encoders, the present disclosure contemplates any suitable stoppers, stop-like systems, or encoders.

[0104] FIG. 8A illustrates the outer ring 805 with notches etched onto the inner surface of the outer ring 805 for a spring-based stop system. The spring 820 is attached to the spring post 810. The retention ring 815 may be composed of Delrin, steel or any other suitable material, and may be segmented or solid/continuous. Figure 8B illustrates an example outer ring with small notches 830 that engage spring-loaded elements to provide tactile feedback from the illustrated stop. In the case of an electronic feedback system, the feedback can be generated in rapid synchronization with the movement of the ring, and must have sufficient launch and decay rates so that successive movements of the ring are distinguishable from each other. In certain embodiments, the outer ring can be freely (eg continuously) rotatable without any clicks or steps. In certain embodiments, based on, for example, an input from the user indicating which rotation mode the outer ring should be in, the ring may be able to rotate both continuously and in steps/increments. Additionally or alternatively, the ring can rotate freely in one direction and in increments in the other direction. Different functions can occur based on the rotation mode used. For example, rotating in continuous mode can change continuous parameters, such as volume or zoom, while rotating in incremental mode can change discrete parameters, such as menu items or contacts in a list, which will be more fully described herein. In certain embodiments, when freely rotating, the ring can provide tactile feedback to the user, such as applying a force to make the ring appear to be in a viscous medium Medium rotation (for example, the faster the ring rotates, the more it resists rotation). In certain embodiments, the outer ring may be depressed or raised in a direction around its axis of rotation, for example as part of a gesture or for changing the rotation mode. In certain embodiments, the outer ring may have a touch sensitive part.

[0105] In certain embodiments, an encoder or a stopper may be used to determine the position of the outer ring relative to the main body of the device. specific

The embodiment utilizes an encoder fixed to the main body of the device, as shown in the encoder 230 of FIG. 2. In a particular embodiment, the encoder is part of the inner surface of the outer ring itself, as shown by printed optics 825 in Figure 8B. In these embodiments, the outer ring directly serves as the rotating part of the encoder. The optical encoder pattern is printed on the inner surface and read by the optical module on the processing board. The encoder inside the outer ring should have sufficient optical contrast for the detector, and can be etched onto the outer ring via, for example, printing or laser etching. A low-friction ring (eg, ring 840 of FIG. 8C) can be used to environmentally seal the inner ring and the outer ring. The low-friction ring is made of materials such as Teflon or Delrin, which prevents contaminants from entering The internal part of the device maintains a tight fit while maintaining a close fit. In certain embodiments, the lip on the inner ring may engage a similar lip on the outer ring, allowing the two rings to be engaged while still allowing free rotation. The larger lip at the bottom of the inner ring provides further sealing by diverting environmental hazards from below. As shown in FIG. 9, in certain embodiments, the sealing ring 915 may be seated in the groove 905 of the base, and the base may include a grip area 910.

[0106] In certain embodiments, the retaining ring connecting the outer ring to the body of the device may have a strain gauge to detect the pressure on the outer ring. As an example, Figure 10 illustrates a retention ring connected to four strain gauges placed symmetrically around the ring (they are also connected to the inner body). As used herein, the four strain gauges can be electronic components that detect strain. Due to the symmetrical placement, normal movement or contact with the outer ring will impose mostly asymmetric strain on the outer ring, because the ring only moves relative to the device in the plane of the ring, so that one end is compressed and the opposite end is elongated, as shown in Figure 10. The top ring is shown. Conversely, squeezing a larger part of the outer ring will likely produce symmetrical strains on the opposite two strain gauges (for example, due to the elongation of the ring under pressure). The relative difference in strain between the two strain gauges distinguishes the deliberate squeeze of the outer ring from the normal movement of the outer ring or the contact with the outer ring. Although the present disclosure describes specific examples of the number and placement of strain gauges in the retention ring, the present disclosure contemplates placing any appropriate number of strain gauges in any appropriate component of the device to detect the pressure on the component. As an example, strain gauges can be placed on the strap of the device or placed in the outer ring.

[0107] When strain is applied to a component containing a strain gauge or any other suitable strain or pressure detection system, the detected strain may result in any suitable function. For example, when strain is applied to the outer ring, such as by the user squeezing the outer ring, feedback may be provided to the user. This feedback can take any suitable form, such as tactile feedback (e.g. vibration, shaking, or heating/cooling), auditory feedback such as beeping or playing a specific user-defined tone, visual feedback (e.g. by the devices display), or Any other appropriate feedback or combination. The functions associated with the squeeze ring will be more fully described herein, and the present disclosure contemplates any suitable function caused by strain or pressure applied to and detected by these components.

[0108] The wearable electronic device may be attached to a strap to secure the device to the user. Here, the reference to "strap" can cover any suitable device for fixing the device to the user, such as a traditional strap 1405 that can be worn on the user's arm, wrist, waist or leg, as shown in the example in FIG. 14A ; Clip 1415 for fixing to a piece of clothing, as shown in the example in Figure 14B; necklace or bracelet 1420 configuration, as shown in the example in Figure 14C; key chain 1425 or other accessory configuration, used to secure the device in For example, in the user's pocket, as shown in the example in Figure 14D; or any other suitable configuration. Each of these embodiments may include a camera 1410 located on the device, on a strap, or on the body. FIG. 11 illustrates various embodiments for wearing the device, for example, on the neck, as shown in 1105 ; Do not go to clothing (for example, the chest shown in 1110); wear on the waist belt as shown in 1115; wear on the appendages (for example, the arm shown in 1120); wear on the arm as shown in 1125 On the waist; or in the pocket as shown in 1130. Although the present disclosure describes specific examples of the way of strapping and securing the device to the user, the present disclosure contemplates any suitable strapping or way of securing the device to the user.

[0109] In certain embodiments, where appropriate, the sensor and corresponding electronics may be attached to the strap. For example, the straps of Figures 14A-14C can be adapted to accommodate optical sensors. As shown, certain embodiments may be adapted to include touch sensitive areas

area. The present disclosure contemplates any suitable straps, which include any suitable sensors or electronic devices, such as communication components (such as antennas), environmental sensors, or inertial sensors. In certain embodiments, the strap can be detached from the device and can communicate with the device remotely when not attached to the device. In certain embodiments, wiring associated with electrical components in the strap may also be contained in the strap, for example, to minimize the volume of the device or minimize electromagnetic interference with internal device components. For example, it may cause high levels of internal EMI (for example, cameras or communication systems), may require additional volume (for example, batteries or speakers), may require environmental sealing of the main body (for example, power/data connectors), or may require Devices with additional contact of the user's skin (eg, biometric sensors) can benefit by accommodating at least some of the electronics in the strap of the device. In certain embodiments, when the wiring is contained in the strap, the display module can be attached to the strap so that the electrical connection made to or via the strap does not twist when the outer ring is rotated. The module can use a user-removable connector, so that the display module or the device body can be removed and attached by the user at will. As an example attachment of the strap to the device, the strap 1215 as shown in FIG. 12A can be attached to the posts by being placed on one or more posts 1205 and then fastened to these posts with fasteners (such as screws) 1210. main body. In certain embodiments, in addition to fasteners and posts In addition, the retaining plate 1215 can also be used to secure the strap to the device 1225, as shown in Figure 12B. This disclosure contemplates any suitable interface between the strap and the device. For example, a USB interface may be provided between the strap and the main body of the device, for example, to transfer data between the device and the strap or a component of the device and a component of the strap. In certain embodiments, the interface may enable the user of the device to easily detach, attach or change the strap of the device.

[0110] The present disclosure contemplates any suitable structure for connecting a strap as shown in FIG. 14A to itself, for example, when worn by a user. For example, FIG. 13A illustrates an example structure of a wearer used to fasten a strap 1305 with a camera module 1310 to the device 1300. The fastener may include one or more buckles 1315, holes 1320 and 1335 and corresponding components, clasps 1340, or clips 1325 with buttons 1330_OFigure 13B illustrates an example mechanism for securing the strap 1301 to the wearer using clips 1311 and 1303. The assembly 1309 is inserted into the cavity on the other side of the assembly 1307 to fasten the strap 130L. FIG. 13B further illustrates an example internal mechanism of the clips 1303 and 1311. The assembly 1317 of the clip 1313 (corresponding to the clip 1311) may include one or more magnetic parts that may be attracted to the magnet in the cavity 1323. For example, the assembly 1317 may include a magnetic portion at its outer edge, and magnets of opposite polarity may be placed in front of the spring 1319 to attract the magnets of the assembly 1317. The assembly 1317 can then fill the cavity 1323, and the clip 1313 is fastened to the clip 1303 through the coupling of the magnet._ο Once inserted, the assembly 1321 can be used to engage the spring 1319, which pushes the assembly 1317 out of the cavity 1323_ΟThe clip 1313 can be detached from the clip 1303. In addition to the magnets on the assembly 1317 and in the cavity 1323, the magnets can also be placed in the clip 1313, for example to assist in removing the clip 1313 when the spring 1319 is engaged or to prevent the assembly 1317 from sliding in when not fastened to the clip 1303 And slide out the clip 1313. For example, one or more magnets can be placed in the center of the clip 1313, equidistant from each component 1317 and in the same plane as the component 1317, with the magnet of each component (and thus the component itself) facing the center of the clip 1313 absorb.

[0111] FIG. 13C illustrates an example structure for fixing the strap 1327 using fasteners 1333 and 1331, for example, by using a cavity 1329 and components 1337 and 1341. FIG. 13C illustrates the internal structure of the fasteners 1331 and 1333. The fastener 1339 (corresponding to the fastener 1333) includes a component 1337. When the fastener 1343 (corresponding to the fastener 1331) is inserted into the fastener 1339, the component 1341 is attached to the component 1337 and can be secured by extending the lip of the fastener 1339. As the fastener 1339 is pulled upward, the lip increasingly pushes the component 1337 out, moving the component 1341 past the lip of the fastener 1339 and enabling the fastener 1339 to be removed from the fastener 1343. In certain embodiments, magnets may be placed in fasteners 1333 and 1331 or on fasteners 1333 and 1331 to fasten them together. For example, a magnet may be placed at the edge of each of the components 1341 and 1337. When the fastener 1343 enters the fastener 1337

In the middle (or vice versa), the magnet attracts and secures the component 1341 to the component 1337. In addition, magnets can be placed in the fastener 1343 to, for example, assist in removing the component 1341 from the component 1337 or prevent the component 1341 from sliding in and out of the fastener 1343 when it is not secured to the fastener 1339_OFor example, one or more magnets can be placed in the center of the fastener 1343, equidistant from each component 1341, and in the same plane as the component 1341, with the magnet at the end of each component (and thus the component itself) facing Attracted by the center of the fastener 1343.

[0112] FIG. 13D illustrates an alternative arrangement for securing the strap 1351 using fasteners 1349 and 1353. When fixed, the fastener 1357 (corresponding to the fastener 1353) can be twisted, releasing the assembly 1359 (which can be round) from the cavity 1363, and allowing the fastener 1361 (corresponding to the fastener 1349) Can be removed from the fastener 1357 and vice versa. In certain embodiments, one or more magnets may be used to secure fasteners 1357 and 1361 to each other and/or remove fasteners 1357 and 1361 from each other. For example, a magnet may be placed in the cavity 1363 and at the outer (convex) edge of the assembly 1359, attracting the assembly 1359 into the cavity 1363 and securing the fastener 1361 to the fastener 1357. As another example, a magnet may be placed on the inner edge of the component 1359 (ie, on the concave surface of the component 1359), attracting the component 1359 into the fastener 1361, for example to assist in removing the component 1359 from the cavity 1363 Or prevent the component 1359 from sliding in and out of the fastener 136L when it is not fastened to the fastener 1357. The corresponding magnet can also be placed on the following surface of the fastener 1361: when the component 1359 is not extended to the cavity 1363 In the middle, these surfaces are in contact with the component 1359. In other words, these magnetic The body can attract (and in certain embodiments eventually directly contact) the magnet on the concave surface of the component 1359, securing the component 1359 to the fastener 1361.

[0113] FIGS. 13E-13G illustrate an example embodiment in which a strap 1369 with a camera module 1373 is fastened to itself when worn by a user of the device 1367, for example. In FIG. 13E, one or more magnets 1371 on one side of the strap 1369 can be attracted to one or more magnets on the other side of the strap 1369. The magnet may be a strip of magnetic material partially passing through the strap, as shown in the magnetic strip 1307 in FIG. 13H, and may be a strip of magnetic material completely passing through the strap, such as the strips 1321 and 1327, or may be a region of magnetic material 1393, as shown in Figure 13F. In addition to the magnets 1371 and 1379, the strap 1369 may also include holes 1391 and one or more posts 1377 for securing the strap 1369 to the wearer of the device 1367. FIG. 13G illustrates a fastener 1387 (eg, a screw 1396) secured to a fastener 1371 (eg, a nut with a shield 1395) to secure the strap 1381 to the wearer of the device 1367 using the hole 1383 (1398).

[0114] In certain embodiments, the strap containing the electrical components may also include a conventional physical contact connector, as shown in the connector 250 of FIG. 2. The connector may allow communication with the device, for example, for charging, system update, debugging, or data transfer. This connector can be of the pogo type or can be a galvanized surface that the charging cable can interface with by contact. Such connectors can be plated with precious metals to prevent corrosion due to exposure to moisture from the environment and the human body. In certain embodiments, a physical connector may be used only for power, and a short-range communication form such as Bluetooth, near field communication (NFC) technology, or WI-FI may be used to transmit data.

[0115] In certain embodiments, the strap may be used to house flexible batteries (eg, lithium-based batteries) to increase the energy storage of the device. Since the energy storage capacity can be associated with the total volume, the battery inside the strap increases the storage capacity of the wearable device with a limited volume without affecting the total size of the device body.

[0116] As described more fully below, a wearable electronic device may include one or more sensors on or in the device. For example, a wearable electronic device may include one or more optical sensors or depth sensors. The optical sensor can be placed in any suitable position, such as placed on the front of the device, placed on the strap, facing the outside from the user's body, placed on the strap, facing the face, placed on the strap, facing the user Body, or any appropriate combination of the above. Figure 15

Illustrated is a banded device 1500 with an outward-facing optical sensor 1505. Placing the optical sensor on the band can reduce the number of high-frequency signals inside the housing, allow lighter shielding in the main body of the device, and thereby allow weight and Save volume. Figures 14A-14D illustrate example camera placements for different embodiments of wearable electronic devices. In certain embodiments, electronic devices such as those used to process camera input may also be located in a strap, for example in a "volcano" shape that houses the camera, as shown in the housing 125 in FIG. 1. In certain embodiments, other sensors may be placed near the optical sensor, for example, placed in the same housing as the optical sensor on the strap of the device. For example, a depth sensor can be used in combination with an optical camera to enhance the display or detection of the environment of the device, or to determine which object the user is pointing at or interacting with via gestures.

[0117] In a specific embodiment, the placement of the optical sensor on the strap can be adjusted by the user within a predetermined range. In certain embodiments, the placement of the optical sensor on the strap can be optimized so that the sensor can be easily aimed by the user. For example, as shown in FIG. 15, if the user wears the device on the user's wrist, the optical sensor 1505 may be placed in an outward-facing manner, so that when the user's palm is approximately parallel to the ground, the optical sensor moves from the user's body to the Aiming outside.

[0118] In certain embodiments, the placement of the optical sensor may allow the user to view the display of the device when the sensor is pointed outward from the user's body. Thus, the user can watch the content captured by the optical sensor and displayed by the device without blocking the user's viewing of the physical scene captured by the sensor, as shown by the viewing triangle in FIG. 16. The display 1620 of the device 1600 may have an associated viewing cone, for example, within which the volume of the display can be reasonably viewed. In FIG. 16, the user 1615 (1) views the real trophy 1610 and (2) views the image of the trophy on the display 1620 of the device 1600 from within the viewing cone of the display 1620 by aiming the sensor 1605 at the real trophy. The sensor 1605 has an associated viewing angle corresponding to the volume within which the sensor 1605 can reasonably capture an image. Note that in the example of FIG. 16, the sensor 1605 is placed so that the user can conveniently aim the sensor 1605 outward while maintaining the display 1620 of the device 1600 in a direction facing the user, and can easily aim the sensor 1605 outward while the device 1600 is not blocked. This is done in the case of the user's viewing of the trophy 1610.

[0119] FIG. 17 illustrates an example viewing angle of an optical sensor. When the object 1725 is in the viewing angle of the optical sensor 1705, the user can view the object 1725 and the image 1710 or 1715 of the object 1725 displayed on the device 1700. For example, when the user's hand 1720 is in the viewing angle, the user can view the object 1725, the hand 1720, and the device's display 1700 ± the object 1725 and the image 1710 of the hand 1720. Conversely, when the hand 1720 is not in the viewing angle of the sensor 1705, the hand 1720 It is not displayed by the image 1715 presented on the display 1700. When worn by the user, the device's sensor can perform a gesture to be captured by the same or other sensors (for example, a gesture to select an object in the device's perspective, such as pinch, tap, or pull or push away). The user's hand/arm/finger is captured in the perspective of the sensor. The sensor and the display can be oriented so that when worn by the user, the object to be displayed on the device is in the viewing angle of the device, while the device does not block the user's view of the object, and the user's gaze is within the viewing cone of the device's display . In certain embodiments, the user can interact with the image captured by the sensor or displayed on the device, for example, by tapping a portion of the display at or near the display image, by performing gestures within the perspective of the sensor, or by any other Appropriate method. This cross Mutual can provide some object-related functions, such as recognizing objects, determining information about objects, and displaying at least some of this information on the display; by capturing images of objects; or if the objects have pairing/communication capabilities, by and Objects pair or communicate with objects in other ways.

[0120] In certain embodiments, the optical or depth sensor modules (they are used interchangeably when appropriate) can be used if the optical sensor is directly mounted on the main printed circuit board (PCB). meeting

A simple extension of the bus used to communicate with the device, as shown in Figure 18A. In FIG. 18A, the optical sensor 1825 sends data to the integrated control 1810 through a flexible printed circuit or wiring 1820. The control 1810 is located in the device 1805 or on the device 1805 in the example of FIG. 18A, and the device 1805 accommodates the main printed circuit board. FIG. 18B illustrates the optical sensor integrated circuit 1850 on or in the optical sensor module 1860, which also houses the optical sensor 1855. The communication between the main printed circuit board of the device 1830 and the electronics in the camera module 1860 takes place via the flexible printed circuit 1845. The arrangement of Figure 18B may allow the integrated circuit to compress and otherwise process data and send data via methods that require fewer signal lines or methods that require smaller data transfers. This may be beneficial because the strap must be flexed when the user wears the device, so fewer lines may be desirable. This solution can reduce the number of lines to one or two signal lines and two power lines, which is advantageous for packaging, molding, and reliability. In certain embodiments, one or more of the aforementioned electronic devices must be shielded to prevent electromagnetic interference from long high-frequency cables. The use of parallel buses is common in this situation and may require the use of larger cables or FPCo

[0121] In one embodiment, the camera control integrated circuit can be directly mounted on a small circuit board at the optical module, as shown in FIGS. 18A-18B. The wearable electronic device may include any suitable sensor. In certain embodiments, one or more sensors or their corresponding electronic devices may be located on the strap of the device, in or on the main body of the device, or both. The sensors can communicate with each other and the processing and memory components through any suitable wired or wireless connection, such as direct electrical connection, NFC or Bluetooth. The sensor can detect the context (for example, environment) or state of a device, a user, an application, or another device or an application running on another device. The present disclosure contemplates that the wearable electronic device includes any suitably configured sensor at any suitable location of the wearable electronic device. Furthermore, the present disclosure contemplates that any suitable sensor receives any suitable input described herein, or initiates, participates in, or is otherwise associated with the provision of any suitable function or service described herein. For example, touch-sensitive sensors can participate in the transition between graphical user interfaces displayed on the device, which will be more fully described in this article. The present disclosure also contemplates that the functions associated with the wearable device, the activation/deactivation of the sensor, the sensitivity of the sensor, or the priority of sensor processing may be user-customizable when appropriate.

[0122] FIG. 19 illustrates an example sensor detection system and illustrates an example sensor for a wearable electronic device. The sensor sends data to the sensor hub subsystem of the device in a sensor-specific format. For example, the sensor 19A shown in the example sensor module 1924 may include one or more: face detection camera 1902, outward facing camera 1904, face proximity sensor 1906, face touch sensor 1908, wearable touch sensor 1910, acoustic skin touch Sensor 1912, inertial measurement system (IMU) 1914, gravity vector sensor 1916, touch sensors 1918 and 1920, and any other suitable sensors 1922. The data from the sensor is sent to the sensor hub 19B shown in the example sensor hub module 1944_OThe data is adjusted as needed in steps 1928 and 1930, noise is removed, and transmitted to the lock state detector 1942. The lock state detector 1942 detects when the device is inactive, and disables the sensor as needed to save power, while monitoring sensor data to obtain a gesture or other appropriate input that can reactivate the device. For example, a numerical gesture detector receives sensor output and compares the output to one or more numerical thresholds to determine the result. The heuristic gesture detector 1934 receives sensor output and makes a decision based on one or more decision trees, such as AND rules applied to more than one threshold. The style-based gesture detector 1938 evaluates the sensor input against a predetermined library of gesture styles 1940, where the gesture style 1940 is, for example, a style determined by an empirical evaluation of sensor output when performing a gesture. One or more gesture priority decoders 1948 evaluate the output from the gesture detector, the lock state detector, or both to determine which of the detected gestures is one-if

Some words one by one should be used to provide functionality to specific applications or system-level processes. More broadly, in certain embodiments, when the device is active, the sensor detectors requested by the application or the system are then activated and provide their data to the sensor priority decoder. In a particular embodiment, the priority detector determines which of the multiple sensor inputs to process-if any, and the present disclosure contemplates that the combined input from multiple sensors can be associated with the input associated with each sensor individually. Functions with different functions. The decoder decides when the sensor is detected with sufficient certainty, and provides sensor data to the sensor hub driver. The driver provides application programming interface (application programming interface, API), the end application and the system controller then generate the necessary output and navigation. For example, Figure 19 illustrates an example sensor hub driver 1950, an application API 1952, a system navigation controller 1954 for determining appropriate system functions, for example (e.g., system-level navigation through the devices graphical user interface 1962), and application 1956's application-level gesture priority detector. Although the sensor hub 19B and the application processor 19C (illustrated in the example application processor module 1964) of FIG. 19 are shown as separate entities, they may be expressed by at least some of the same or similar components (and their functions may be At least some of the same or similar components). In certain embodiments, the boundaries between the components and functions of the sensor hub and the application processor may be more or less inclusive. The boundary shown in FIG. 19 is only an example embodiment. As for the sensor itself, the functions performed by the sensor hub system and the application processor and their components can occur in the main body of the device, in the strap, or in both, or in the device body, in the strap, or in the main body of the device. Of the two. Certain embodiments may use more than one sensor hub or application processor, or components thereof, to receive and process sensor data.

[0123] The sensor can generate sensor data internally, and the sensor data can be simply filtered or reformatted by, for example, a detector or a data conditioner. The raw data can be formatted into a unified format by the data formatter for ingestion by the application API. The recognizer may use numerical models (such as decision trees), heuristic models, pattern recognition, or any other appropriate hardware, software, and technology to detect sensor data, such as gesture input. The recognizer can be enabled or disabled by the API. In this case, if the recognizer does not receive data from the sensor or cannot recognize the sensor data, the associated sensor can also be disabled.

[0124] The device may contain a database of sensor output that allows the same detector to detect many different sensor outputs. Depending on the request generated by the API, the sensor priority decoder can suppress or pass the sensor output based on the provided criteria. The standard may be a function of the design of the API. In certain embodiments, the recognizer may ingest the output of more than one sensor to detect the sensor output.

[0125] In certain embodiments, multiple sensors may be used to detect similar information. For example, both ordinary and depth-sensing cameras can be used to detect fingers, or both gyroscopes and magnetometers can be used to detect orientation. When appropriate, functions that depend on or utilize sensor information can be based on implementation and runtime considerations, such as cost, energy use, or frequency of use, to replace the sensor or make a choice among them.

[0126] The sensor may be of any suitable type, and as described herein, may be located in or on the main body of the device, in or on the strap, or a suitable combination of the above. In certain embodiments, the sensor may include one Or multiple depth or proximity sensors (terms used interchangeably as appropriate in this document), such as infrared sensors, optical sensors, acoustic sensors, or any other suitable depth or proximity sensors. For example, a depth sensor can be placed on or near the display of the device to detect when, for example, the user's hand, finger, or face is close to the display. As another example, the depth sensor can detect any object pointed by the user's finger in the perspective of the depth sensor, which will be more fully described in this article. Additionally or alternatively, the depth sensor may be located on the strap of the device, which will be more fully described herein. In certain embodiments, the sensor may include one or more touch sensitive areas on the device body, strap, or both

area. The touch-sensitive area may utilize any suitable touch-sensitive technology, such as resistance, surface acoustic wave, capacitance (including mutual capacitance or self-capacitance), infrared, optical, dispersion, or any other appropriate technology. The touch sensitive area can detect any appropriate contact, such as swiping, tapping, touching one or more specific points or contact with one or more specific areas, or multi-touch contact (for example, pinching two or more points on the display). Multiple fingers or rotate two or more fingers on the display). As will be described more fully herein, the touch sensitive area may include at least a portion of the display, ring, or strap of the device. As with other sensors, in certain embodiments, touch-sensitive areas may be activated or deactivated based on, for example, context, power considerations, or user settings. For example, the touch-sensitive part of the ring can be activated when the ring is "locked" (eg, not rotating) and deactivated when the ring is free to rotate. In certain embodiments, the sensor may include one or more optical sensors, such as a suitable camera or optical depth sensor.

[0127] In certain embodiments, the sensors may include one or more inertial sensors or orientation sensors, such as accelerometers, gyroscopes, magnetometers, GPS chips, or compasses. In certain embodiments, the output from the inertial or orientation sensor can be used to activate or unlock the device, detect one or more gestures, interact with the content on the display screen of the device or the display screen of the paired device, access specific data or activate the device or The specific function of the paired device, initiation of the communication between the main body of the device and the band or the device and the paired device, or any other appropriate function. In certain embodiments, the sensor may include one or more microphones for detecting, for example, the user's voice or environmental sound to determine the context of the device. In addition, in certain embodiments, the device may include one or more speakers on the device body or on the strap.

[0128] In certain embodiments, the sensor may include a sensor used to communicate with devices such as network devices (such as servers or routers), smart phones, computing devices, display devices (such as televisions or digital kiosks), audio systems, video systems, and other wearable devices. Components used for communication with other devices such as electronic devices or for communication with the main body of the device. Such sensors may include NFC readers/beacons, Bluetooth technology, or antennas for transmission or reception at any appropriate frequency.

[0129] In certain embodiments, the sensor may include a sensor that receives or detects tactile input from the user of the device, such as a piezoelectric body, a pressure sensor, a force sensor, an inertial sensor (as described above), a strain/stress sensor, or a mechanical sensor. Actuator. Such sensors can be located in any suitable location on the device. In certain embodiments, the components of the device may also provide tactile feedback to the user. For example, one or more rings, surfaces, or straps can vibrate, generate light, or generate audio.

[0130] In certain embodiments, the wearable electronic device may include one or more sensors of the surrounding environment, such as a temperature sensor, a humidity sensor, or an altimeter. In certain embodiments, the wearable electronic device may include one or more sensors for sensing the physical attributes of the user of the wearable device. Such a sensor can be located in any suitable area, such as on the strap of the device, or on the base of the device, in contact with the user's skin. As an example, the sensor may include an acoustic sensor, which detects the user's skin when the user rubs the skin near the wearable device (or clothes covering the skin), taps the skin near the device, or moves the device up and down on the user's arm. vibration. As another example, the sensor may include one or more body temperature sensors, pulse oximeters, galvanic skin response sensors, capacitive imaging sensors, electromyography sensors, biometric data readers (such as fingerprints or eyes), and any other suitable Sensor. This sensor can provide users with feedback on the users status, can be used to initiate predetermined functions (such as reminders to take specific medications, such as insulin for diabetes), or can transmit the sensed information to remote devices (such as in a medical office). Terminal).

[0131] The wearable electronic device may include one or more charging components for charging or powering the device. The charging component can use any appropriate charging method, such as capacitive charging, electromagnetic charging, trickle charging, charging by direct electrical contact, solar energy, kinetic energy, induction or smart charging (for example, charging based on the condition or state of the battery, and charging accordingly Modify the charging action locally). The charging component can be located on any appropriate part of the device, such as in or on the main body of the device or

In or on the equipment belt. For example, FIG. 20A illustrates a charger 2000 having a socket 2005 for connecting the charging assembly with the charger. For example, the socket 2005 can use friction, mechanical structures (such as latches or buckles), magnetic force, or any other appropriate technology to accept and secure the tooth head from the charging assembly so that the tooth head and the charger 2000 make direct electrical contact. . FIG. 20C illustrates that the gear head 2015 on the strap 2010 uses a pogo-type connector to create a circuit connection between the charger 2022 and the strap 2010 through the contacts 2020. In a particular embodiment, the tooth head 2015 may be on the charger 2022 and the slot 2005 of FIG. 20A may be on the body of the strap or wearable device. In certain embodiments, the contacts 2020 (eg, pogo-style connectors) can be on the body of the device, which can be used to create a circuit between a strap or a charger used to charge the device. The charger 2000 of FIG. 20A can be connected to any suitable power source (for example, power from an alternating current (AC) socket or direct current (direct current, direct current, direct current) from a USB port on a computing device) through any suitable wired or wireless connection DC) Electricity).

[0132] The charger 2000 may be composed of any suitable material, such as acrylic, and in certain embodiments may have a non-slip material as its backing, such as rubber. In certain embodiments, the charger 2000 may be fixed or attached to a surface, for example, may be attached to a wall, as shown in FIG. 20B. The attachment can be performed by any suitable technique, such as mechanically, magnetically or adhesively. In certain embodiments, the wearable electronic device can be fully usable while being attached to the charger. For example, when the charging component is located on the main body of the device, the device can be located in the charger while the user interacts with the device or other devices communicate with the device.

[0133] As another example of a charging component in a wearable electronic device, FIGS. 21A-21B illustrate an additional example charger using, for example, an inductive charger. As shown in Figures 21A-21B, the strap may include one or more charging coils 2110. As described above, instead of or in addition to the strap of the device, the present disclosure contemplates being included in the device The charging coil (or any other suitable charging component) in or on the main body. The magnetic field 2105 generated by, for example, the charging surface 2115 or the charging surface 2120 passes through the charging coil 2110. The charging surface 2120 of FIG. Placed, thereby increasing the charge transfer rate of the system. The present disclosure contemplates that when appropriate, charging can power components in or on the main body of the device, components in or on the strap, or both.

[0134] In certain embodiments, a band or device may implement an antenna for a wireless charging scheme. Since wireless charging operates optimally in the absence of ferrous metals, this allows a wider choice of materials for the body of the device, while at the same time by allowing the coil to be held between the poles of the charging driver (as described above) Rather than simply being coplanar with the drive to allow for increased wireless charging transfer capacity. As described above and as shown in FIG. 2, the active strap may also include a conventional internal physical contact connector 250.

[0135] In a particular embodiment, a charging unit with an internal charge storage may be associated with a wearable electronic device. When inserted into the wall, the charging unit can charge both the attached equipment and the internal storage of the charging unit. When not plugged in, the charging unit can still charge the attached equipment from its power storage until the storage is exhausted. When only the charger is connected to the power source and there is no device, it still charges itself so that it can provide additional power to the device at some time in the future. Thus, the charging unit described herein is useful when it is plugged in and not plugged into the power source, because when a person cannot connect to the power source, such as when traveling, on an airplane, at a train station, outdoors, or at a user When the device needs to be charged but cannot be connected to the power source, the charging unit can also supply power for any part of the charged device for a period of time. When the charger is charging the device, the device can be in standby or in use, and there is no need to modify the software or hardware of the target device. Additional benefits of one or more embodiments of the present invention may include reducing the number of items a person must carry, providing the benefits of both a charger and a power pack, making it useful to carry a charger while on the move, and reducing one

The number of cables and connectors that people must carry in order to extend the battery life of their devices. The present disclosure envisages that this charging unit can be applied to any appropriate electronic device, including but not limited to wearable electronic devices.

[0136] FIGS. 22A-22B illustrate specific embodiments of an example charging unit 2210 with an example connection 2205 to a device 2200 and connections 2215 and 2220. For example, FIG. 22A illustrates cable connection from the charging unit 2210 to the device 2200 and to the external power source. As another example, FIG. 22B illustrates a charging unit 2210 with cable connection from the device 2200 and direct connection to the power source. The present disclosure contemplates any suitable connection between the device, the charging unit, and the power source for charging the charging unit. For example, the connection to both the device and the power source can be direct, via cable, or wireless.

[0137] As described above, the charging unit can charge the device from the internal charging warehouse of the charging unit even when it is not connected to the external power source, and can charge itself, the connected device, or both when connected to the external power source.Bycharges. The present disclosure contemplates any suitable scheme for distributing charge between the charging unit and the device. This distribution scheme may depend on the amount of charge inside the device, the amount of charge inside the charging unit, the amount of power the device is consuming, the charging capacity of the external power source, or any appropriate combination of the foregoing. As an addition or alternative, the charging threshold may determine which allocation scheme is used. For example, one charging scheme can be used when the device is close to full charge and there is little charge left in the charging unit, and another charging scheme can be used when the device has little charge left. Figures 23-24 illustrate example charging schemes for the charging unit and connected devices. For example, as shown in FIG. 24, when the device is connected to the charger as in step 2400, step 2405 determines whether the device is fully charged. If it is, no further charging action is taken. If not, step 2410 determines whether the charger is connected to an external power source, such as line voltage. If so, the device is charged from the external source in 2425. If not, the step determines whether the charger has any power left. If yes, the device is charged from the internal power supply of the charger in step 2420, and when the charging unit is connected to the line voltage, the charging unit is charged from the line voltage instead of the charging unit. Warehouse Standby for charging. Figure 23 illustrates a similar decision tree. If the device is connected to a charger (step 2300) connected to a power source (step 2300), step 2310 determines whether the device is fully charged, and if not, then the device is charged from the power source to which the charger is connected (step 2315) ο similar Specifically, step 2320 determines whether the charger is fully charged, and if not, then in step 2325 the charger unit is charged from the power source. In certain embodiments, the allocation scheme used can be determined or customized by the user.

[0138] FIGS. 25A-25E illustrate example embodiments of energy storage and charging and charging units in a device. In FIG. 25A of the illustrated embodiment, the charge storage 2500 of the device and the charge storage 2520 of the charging unit are both depleted. 25B-25C illustrate charging the charge storage 2500 of the device and the charge storage 2505 of the device after the charging unit is connected to the external power source 2510. After a short time, both the charging unit and the device are charged at the same time, and the distribution of the charging is such that each is given the same percentage of its total charge capacity. The charging warehouse 2500 of the equipment and the charging warehouse 2505 of the charging unit are both fully charged after a period of time, as shown in FIG. 25C. As described herein, the amount of charge distributed to the device or charging unit may vary based on any suitable charge distribution scheme. For example, if the power conversion capacity of the charging unit is limited, the storage of the charging unit is almost full and the charge storage of the device is almost empty, or the energy demand of the device is very high, the charging unit can give priority to charging its internal storage Device charging ο As another example, the charging of the charging unit may continue until a predetermined threshold charge is reached.

[0139] FIGS. 25D-25E illustrate charge transfer between the charging unit and the device when the charging unit is not connected to an external power source. As shown in FIG. 25D, a device with little charge remaining in its warehouse 2500 is connected to a charging unit with a fully charged warehouse 2505. As described above, the present disclosure contemplates any suitable charge distribution scheme between the device and the charger when the charger is not connected to an external power source. This distribution scheme can be used when the charging unit is connected to an external power source

The distribution plan is the same or different. For example, FIG. 25E illustrates a distribution scheme that maximizes the charge of the charging warehouse 2500 of the device. As long as the charging unit still has a charge, it continues to charge the device until the device is fully charged or until the charging warehouse 2505 of the charger is completely empty.

[0140] FIG. 26 illustrates an example internal architecture of an example charging unit 2600. The line voltage converter 2605 generates a lower voltage direct current from the high voltage line current 2610. This voltage is fed to both the battery charger/regulator 2630 and the connector 2615 to which the device can be connected via connection 2620 for charging. The battery charger 2630 uses the available power from the line voltage converter 2605 to charge the energy warehouse (battery 2635). It can obtain the same share of power as the equipment, and obtain a smaller share when the equipment demand is high (equipment priority) or a larger share when the internal power reserve is low (charger priority). These priorities can be user selectable.

[0141] Continuing the example of FIG. 26, when the line voltage converter 2605 does not provide power, the charger/regulator 2630 generates an appropriate charging voltage from the power on the battery 2635. The regulator 2630 may be turned on at all times, or it may be turned on by a connection with the device or by pressing a button that indicates that the user wishes to charge the device. Once activated, the regulator 2630 will charge the device until the internal reserves are exhausted. At this time, some charge may still remain in the battery 2635 to improve battery life, but it will not be available to the user. The device may include an emergency mode to allow access to some of this energy to obtain a minimum measure of emergency use time, at the expense of battery life. The regulator 2630 can continue to provide energy until the device is unplugged, or until the device only absorbs a minimum amount of energy, which indicates that the charging is complete. Finally, the charger/regulator 2630 may include an on-demand display that shows the user the amount of energy remaining in the reserve. Since displays generally use energy, buttons or other inputs can be used to trigger the display for a limited time. Although FIG. 26 illustrates an example internal architecture of an example charging unit 2600, this disclosure contemplates any suitable internal architecture of any suitable charging unit described herein, and it is envisaged that such a charging unit may have any suitable size and shape.

[0142] In certain embodiments, functions or components of the device (such as sensors) can be activated and deactivated, for example, to save power or reduce or eliminate unwanted functions. For example, the lock state detector detects when the device is not activated, and disables the sensor as needed to save power, while monitoring sensor data to obtain gestures or other appropriate inputs that can reactivate the device. The device may have one or more power modes, such as a sleep mode or a fully active mode. As an example, in certain embodiments, the device is arm-worn, and the touch surface of the device can come into contact with objects and people during normal use. To prevent accidental activation, accelerometers or other inertial sensors in the main body or strap of the device can be used to determine the approximate position of the device relative to the earth's gravity. If it is detected that the gravity vector is toward the side of the device (for example, it is determined that the device is on the side of the user or the display is not pointed at the user), the touch screen can be locked and the display can be disabled to reduce energy usage. When it is determined that the gravity vector is pointing below the device (for example, the device is roughly horizontal, leading to the determination that the user is watching or using the device in other ways), the system can power up the display and enable the touch screen for further interaction. In certain embodiments, as an addition or alternative to the direction of the gravity vector to wake up or unlock the device, the change rate of the direction or amplitude of the gravity vector can be used to wake up or unlock the device. For example, if the rate of change of the gravity vector in a predetermined amount of time is Zero (in other words, the device has been held in a specific location for a predetermined amount of time), the device can be awakened or unlocked. As another example, one or more inertial sensors in the device may detect a specific gesture or sequence of gestures used to activate the display or other appropriate components or applications. In certain embodiments, the encoder of the device is robust to accidental activation, allowing it to be left active so that the user can change between choices while lifting the device into its perspective. In other embodiments, the encoder may be deactivated based on context or user input.

[0143] As an addition or alternative to power saving, certain embodiments may lock one or more sensors, specific functions, or specific applications to provide security for one or more users. Appropriate sensors can detect the

The security activation or unlocking of another device for pairing or communication. For example, one or more security aspects of the device can be unlocked with specific gestures performed by the device or performed on the touch-sensitive area of the device. As another example, a particular rotation or sequence of rotations of the rotatable ring of the device may unlock one or more security aspects of the device alone or in combination with other user input. For example, the user can turn the rotatable ring to a unique sequence of symbols such as numbers or pictures. In response to receiving a sequence of rotation inputs for rotating the rotatable ring, the display may display specific symbol(s) corresponding to each rotation input, which will be more fully described herein. In certain embodiments, the symbols used may be user-specific (for example, a user picture stored on the device or accessible to the device, or a symbol pre-selected by the user). In certain embodiments, different symbols may be presented to the user after a predetermined number of unlocks or after a predetermined amount of time. The example inputs described above may additionally or alternatively be used to activate/deactivate some aspects of the device, specific applications, or access to specific data. Although the present disclosure describes specific examples of user input to unlock the security aspect of the device, the present disclosure contemplates any suitable input or combination of inputs for unlocking any security aspect of the device. The present disclosure contemplates that inputs or other appropriate parameters for unlocking the security aspects of the device or activating/deactivating components of the device may be user-customizable.

[0144] In certain embodiments, the wearable electronic device may detect one or more gestures performed with or on the device. The gesture can be of any suitable type, can be detected by any suitable sensor (such as an inertial sensor, touch sensor, camera, or depth sensor), and can be associated with any suitable function. For example, one or more depth sensors can be used in conjunction with one or more cameras to capture gestures. In certain embodiments, several depth sensors or cameras can be used to enhance the accuracy of detecting gestures or the background associated with the gestures. When appropriate, sensors for detecting gestures (or for initiating processing of functions associated with gestures) can be activated or deactivated to save power or provide security, which is more fully described above. As shown above, Figure 19 illustrates an example sensor detection system and provides specific examples of gesture detection, processing, and prioritization. In certain embodiments, a certain application can reserve certain gestures or reserve all available gestures; or the user can select which gestures should be detectable by which applications. In certain embodiments, the gesture may include the manipulation of another device while using the wearable device. For example, gestures may include shaking another device while aiming, moving, or otherwise utilizing the wearable device. The present disclosure contemplates that any gesture described herein may involve manipulation of another device when appropriate. Although the examples and illustrations discussed below involve specific aspects of gestures or Attributes, but the present disclosure contemplates combining any suitable aspects or attributes of the gestures and sensors described herein.

[0145] In certain embodiments, the wearable electronic device may detect one or more gestures performed with or on the device. The gesture can be of any suitable type, can be detected by any suitable sensor (such as an inertial sensor, touch sensor, camera, or depth sensor), and can be associated with any suitable function. For example, one or more depth sensors can be used in conjunction with one or more cameras to capture gestures. In certain embodiments, several depth sensors or cameras can be used to enhance the accuracy of detecting gestures or the background associated with the gestures. When appropriate, sensors for detecting gestures (or for initiating processing of functions associated with gestures) can be activated or deactivated to save power or provide safety, which is more fully described above. As described more fully above, FIG. 19 illustrates an example sensor detection system and provides specific examples of gesture detection, processing, and prioritization. In certain embodiments, a certain application can reserve certain gestures or reserve all available gestures; or the user can select which gestures should be detectable by which applications. In certain embodiments, the gesture may include the manipulation of another device while using the wearable device. For example, gestures may include shaking another device while aiming, moving, or otherwise utilizing the wearable device. The present disclosure contemplates that any gesture described herein may involve manipulation of another device when appropriate. Although the examples and illustrations discussed below involve gestures Specific aspects or attributes, but this disclosure contemplates any suitable aspects or attributes that combine the gestures and sensors described herein

sex.

[0146] In certain embodiments, gestures may include one-by-one gestures involving at least one hand of the user and the appendage wearing the device, for example, the other wrist of the user. For example, in certain embodiments, the user can use the hand/arm wearing the device to appropriately aim the optical sensor of the device (such as a camera or depth sensor) and can move or position the other arm/hand/finger to perform a specific gesture. As described herein and as shown in Figures 16-17, in certain embodiments, the sighted scene can be displayed on the display of the device so that the user can view the real scene, the scene displayed on the device, and the users hand/ The arms/fingers one by one if they are in the viewing angle. In a particular embodiment, the displayed scene may include the hand/finger/arm used to perform the gesture detected by the sensor. Figures 27-28 illustrate example gestures in which the user aims at a sensor on the device (such as on the strap of the device, as shown in the figure) outward (such as away from the user's body) sensor and moves or positions his other arm /Hand/finger to perform gestures. For example, in FIG. 27, the outward sensor detects the object 2705 in the perspective of the sensor, and the outward sensor (which may be the sensor that detects the object) detects one or more fingers 2710 pointing at the object, and when pointing ( When one or more fingers are determined to be stationary 2715, a gesture 2720 is detected. Referring to Figure 19, the raw gesture data captured by the outward camera can be adjusted and noise removed. Sound and the data can be sent to the probing gesture detector. The gesture priority decoder processes the gesture data and determines when the gesture is recognized with sufficient certainty. When a gesture is recognized, the gesture is sent to the sensor hub driver, which provides APIs to end applications and system controllers.

[0147] As an example of the function associated with this gesture, the camera can focus on an object, the detected and pointed object can then appear on the display, information about the object can appear on the display, and the displayed The content can be transferred to the display of another device (for example when the object is another device). FIG. 28 illustrates an example gesture similar to that of FIG. 27; however, the illustrated gesture includes an outward sensor detecting a "tap" movement of the finger (one or more) (for example, the finger (one or more) Moving away from the sensor). For example, the gesture of FIG. 28 may include detecting an object in the scene of the camera (or other appropriate sensor) in step 2805, detecting the finger in the scene in step 2810, and detecting the lack of lateral movement of the finger in step 2815. , In step 2820, it is detected that the fingertip moves further away from the sensor, and in step 2825, a gesture is detected. The gesture shown in Figure 28 may provide any suitable function. For example, the object to be "tap" can be selected from the objects displayed on the display screen.

[0148] FIGS. 29-30 illustrate example gestures in which an object is detected with an outward sensor following the movement of the user's fingers and hand. For example, FIG. 29 illustrates that the outward sensor detects that two fingers are separated 2915, the two fingers come together (for example, in a pinch movement) 2920, and then the pinched fingers move 2925 toward the sensor. The motion of the fingers coming together and moving towards the sensor can occur simultaneously or sequentially, and performing these steps sequentially (or the time between steps in a sequence) or performing these steps simultaneously can each be a different gesture. In FIG. 30, the two fingers illustrated are initially close together 3030, and the outward sensor detects that the fingers are separated 3020 and the hand 3015 is moved away. For Figure 30, the movement of the fingers and hands can be simultaneous or in any appropriate sequence. In addition, some aspects of Figures 29-30 can be combined to form a gesture. For example, pinching the fingers together and moving away from the sensor can be a unique gesture. In certain embodiments, the detected finger or hand may manipulate another device, and this manipulation may form part of the gesture. For all the example gestures described herein, this disclosure contemplates any suitable functions associated with the gestures shown in FIGS. 29-30.

[0149] FIGS. 31-32 illustrate example gestures similar to FIGS. 29-30, except that all fingers are used to perform the gesture here. In FIG. 31, the fingers are detected as being close together initially (for example, making a fist) 3105, the fist is detected to move away from the sensor 3110, and the sensor detects that the fist is open 3115. Likewise, the sequence of steps illustrated can occur in any suitable order. FIG. 32 illustrates the inversion of FIG. 31. Figures 31-32 can be associated with any suitable function. For example, Figure 31 illustrates

The following example is given: sending all or part of the content displayed on the device to another device, such as the television shown in FIG. 31. Similarly, the gesture in FIG. 32 can pull some or all of the content displayed on another device onto the display of the wearable device. For example, the gestures of Figures 31-32 can be implemented when the user uses a wearable device to perform gestures near another device, such as a smart phone, a tablet device, a personal computing device, or a smart home appliance (such as a refrigerator, thermostat, or Washing machine) or any other appropriate equipment. The described functions are only examples of functions that can be associated with the gestures shown in FIGS. 31-32, and the present disclosure contemplates that other appropriate gestures can perform the described functions.

[0150] FIGS. 33-37 illustrate the outward sensor detecting a part of a hand or arm swiping in front of the sensor. In certain embodiments, swiping with the front of the hand and swiping with the back of the hand may be different gestures. Figures 33-34 illustrate the perspective of the hand swiped across the sensor from right to left 3310-3315 and from left to right 3410-3415, and Figures 35-37 illustrate the hand being swiped from bottom to top 3510-3515 (and 3735-3740) and swept across the view of the sensor from top to bottom 3610-3615 (and 3710-3715). As shown in the figure, the hand can initially start in the angle of view, pass through the angle of view and leave the angle of view (as shown in Figure 36); it can start outside the angle of view, pass through the angle of view and leave the angle of view (as shown in Figure 37); it can start at Outside the angle of view, pass through a part of the angle of view and remain in the angle of view (as shown in Figures 33-35); or it can start in the angle of view, pass through a part of the angle of view and remain in the angle of view. The present disclosure contemplates that the hand is swiped at other angles, such as entering at a 45-degree angle below and exiting to the right of the device and at a 45-degree angle relative to the upper left of the device. In addition, the present disclosure contemplates detecting hand sweeps that are in motion instead of straight lines, such as curved sweeps or triangular sweeps. This disclosure contemplates any appropriate function associated with any or all of the gestures shown in Figures 33-37, such as between user interfaces displayed on the device or when active and displayed on Switch between applications on the device, open or close applications, or scroll through displayed content (such as documents, web pages, or images). As reiterated elsewhere, this disclosure contemplates any suitable gestures associated with the functions described in connection with Figures 33-37. [0151] FIGS. 38-39 illustrate example gestures in which the outward sensor detects that the user's hand is in the field of view 3805 and detects that one or more fingers point in a direction (in certain embodiments together with the user's hand or arm Part) 3815. The detected gesture may depend on the detected finger or the direction in which the detected finger is pointing. For example, as shown in FIG. 38, the finger may be a thumb 3820 pointing upward, and in FIG. 39 the finger may be a thumb 3920 pointing downward. Any appropriate function can be associated with the gestures shown in Figures 38-39, such as saving or deleting files locally on the device or on an associated device, or approving or disapproving changes to settings or other content.

[0152] FIG. 40 illustrates an example gesture involving a shape made with multiple fingers or a part of a hand in the perspective of the outward sensor. As shown in FIG. 40, the shape may be a ring 4010, and the gesture may include a finger not involved in the shape pointing in a specific direction 4015_oAs shown in Figure 40, the gesture may include maintaining the shape 4020 (and possibly other fingers) for a predetermined amount of time.

[0153] FIGS. 41-42 illustrate example gestures that include covering all or part of the outward sensor with the user's finger or hand. Using a thumb-down gesture to cover the sensor 4105 from the top of the device (as shown in FIG. 41) and from the bottom 4210 of the device (as shown in FIG. 42) or to cover the sensor from the side of the device may be different gestures. The direction of coverage can be detected by, for example, the shape of the hand when covering the device, the orientation of the hand when covering the device, data from other sensors indicating the direction of covering the outward sensor (such as detecting that the display and the outward sensor are covered), or any other appropriate technology .

[0154] FIG 43-44 illustrates graphs embodiment gesture, wherein a portion of the user's one or more fingers or a hand / arm at the angle of view is detected 4305/4405 outward sensor, and the angle of view (or "frame ") to perform a specific gesture 4310/4320/4410/4420. In a specific embodiment, the gesture may be any suitable motion or a specific style of motion. In certain embodiments, the gesture may be associated with a detected finger or part of the hand/arm. For example, single finger

The holding finger may be associated with a gesture 4305 (as shown in FIG. 43) or multiple fingers/palms may be associated with a gesture 4405 (as shown in FIG. 44). In certain embodiments, the direction of the palm (eg, front, back, at an angle) can be detected and associated with the gesture.

[0155] FIG. 45 illustrates an example gesture, including detecting a shape 4505 made with multiple fingers or hands/arms of the user, and detecting the movement of the shape in the viewing angle 4510/4520. FIG. 45 illustrates the shape of FIG. 40 in The outward sensor moves through the entire field of view.

[0156] FIG. 46 illustrates an example gesture that involves detecting one or more fingers (some or all of the user's hand/arm) and their initial orientation, and then detecting the change in orientation or the rate of change in orientation over time . For example, Figure 46 illustrates that two fingers are detected in the angle of view in step 4605, these fingers and the edge of the hand are detected in the angle of view in step 4610, and the fingers are detected in a "C" shape in step 4615. The initial orientation of the "C" shape is decoded, the orientation change of the "C" shape is decoded in step 4625, the relative rotation value of the "C" shape is determined in step 4630, and the gesture is detected in step 4635. The present disclosure contemplates any suitable shape made with the user's fingers/hands/arms.

[0157] FIG. 47 illustrates an example gesture that involves detecting the number of fingers at a specific position in the angle of view of the outward sensor. For example, FIG. 47 illustrates the detection of fingertips, such as an extended thumb, an extended thumb and one finger, an extended thumb and two fingers, in a viewing angle at step 4705. In step 4710, the specific fingertip orientation configuration is detected, and in step 4715, the mapping of the configuration to at least the numerical count of the finger is performed to detect the gesture in step 4725. Each displayed image can be a different gesture. The present disclosure contemplates any suitable position of the fingers that make up the gesture. As for all other example gestures described herein, this disclosure contemplates any suitable functions associated with these gestures. For example, each gesture of FIG. 47 can be associated with a contact to call, email, or text, and the detected gesture can activate a call, email, or text to the contact assigned to the gesture. In certain embodiments, the position of the hand/arm/finger may indicate which contact method should be used for the contact associated with the gesture.

[0158] FIGS. 48-49 do not illustrate example gestures involving dual sensors on the device. For example, Figure 48 does not show the sensor on the bottom part of the device. The sensor detects the position of the user's other hand relative to the device, and detects the separation of the hand from the sensor. In certain embodiments, the gesture may include determining that both hands are moving, such as by additional information provided by one or more inertial sensors in the device, or by detecting the introversion of the device's movement through changes in the scene (eg, facing the user Body) camera to determine. For example, in FIG. 48, at step 4805, a hand is detected in the angle of view. The sensor detects that the hand is in a pinch shape in step 4810, and the same sensor or another sensor detects that the device is in a horizontal orientation in step 4815. The sensor detects hand movement relative to the device in step 4820 and estimates the relative position in step 4825. At step 4830, a gesture is detected. Similarly, Figure 49 illustrates an example gesture that also involves detecting that the user's hand is in view and then moving away from the device sensor. However, in Figure 49, the device sensor is positioned on the top of the device (for example, the forward sensor). As an example, in step 4905, a hand is detected in the perspective of the forward camera. In step 4910, it is detected that the hand is in a pinch shape, and in step 4915, it is detected that the device is in a horizontal orientation. In steps In 4920, the hand moves closer or farther away from the device, and relative position estimation is performed in step 4925, at which time a gesture is detected in step 4930.

[0159] FIGS. 50-58 illustrate example gestures detected by at least one forward-facing sensor, such as a sensor on the top of the device. When appropriate, any gesture of FIGS. 50-58 can be detected by a sensor in any other suitable location (for example, facing outward, as described above), and any gesture detected by a sensor described in another location can be detected by a forward sensor detected. Figure 50 illustrates an example gesture involving one or more fingertips hovering over the device, and the forward sensor detects the fingertip in step 5005, and detects the position of the fingertip in steps 5010 and 5015 or these Fingertips

Movement (or lack of movement) to detect a gesture in step 5020. Figure 51 illustrates an example gesture, where steps 5105 and 5110 are the same as 5005 and 5010, respectively. However, the detected fingertips are moved away from the front sensor in step 5115; in certain embodiments, the gesture may include detecting that one or more of the fingertips change positions relative to each other, such as being separated in step 5120. Figure 52 illustrates that the sensor detects the fingertips in step 5205, the fingertips are moved together in step 5210, the fingers are moved toward the device in step 5215, and the duration of the movement is detected in step 5220 to be detected in step 5225 To the gesture. As shown in FIG. 53, in certain embodiments, in addition to the movement of the fingertip toward the sensor, the gesture may also include detecting a change in the relative position of the fingertip. For example, in step 5305, one or two fingers are detected on the front surface; in step 5310, the fingers are detected moving up or down; and in step 5315, a gesture is detected. In certain embodiments, the duration of the gestures of FIGS. 50-52 may determine whether a gesture is detected, or different durations may constitute different gestures.

[0160] FIGS. 54-57 illustrate example gestures involving the movement of one or more fingers or part of a hand/arm across the front of the device (and thus across the forward sensor). As shown in the figure, gestures can depend on the number of fingers used (for example, two fingers and the entire palm); depend on the direction of movement across the front of the device (for example, from bottom to top or left to right); depending on the cross device The duration of the frontal movement; depends on the proximity of the detected finger or hand/arm to the front of the device; depends on the part of the front of the device (for example, all or part of it, and the relative position of the part (for example, the lower half)); Or it depends on whether the detected part is initially in the viewing angle of the front sensor, initially outside the viewing angle, ending in the viewing angle, or ending outside the viewing angle. For example, the gesture of FIG. 54 may include detecting one or two fingers detected on the front surface in step 5405; detecting a finger moving to the left in step 5410, and detecting a gesture in step 5415. As another example, FIG. 55 may include one or two fingers detected on the front surface detected in step 5505; rightward movement of the fingers is detected in step 5510, and gestures detected in step 5515. As another example, FIG. 56 may include that no fingers are detected in step 5605, multiple fingers are detected to enter the viewing angle from the left, and the front surface is detected to be covered. In step 5620, it is detected that the finger leaves the picture frame, and in step 5625, the gesture is detected. As another example, FIG. 57 may include that no fingers are detected in step 5705, multiple fingers are detected to enter the viewing angle from the right in step 5710, coverage of the entire front surface is detected in step 5715, and the detection in step 5720 When the finger leaves the view, the gesture is detected in step 5725. As with all gestures described herein, any appropriate combination of these factors (and any other appropriate factors associated with the gesture) can be used to determine the gesture or the function corresponding to the gesture. Any suitable function may be associated with gestures, for example, transitioning between graphical user interface screens, scrolling through displayed content, or scrolling through available applications or devices to communicate/pair with them.

[0161] FIG. 58 illustrates an example gesture involving the detection of one or more fingers at the edge of the device, and may include movement of these fingers around all or part of the edge of the device. For example, as shown in Figure 58, the gesture may include not detecting a finger in step 5805, detecting a single finger at the edge of the front in step 5810, and detecting that the finger moves along the edge in step 5815. In 5820, the angular movement of the finger relative to the device is decoded, and in step 5825, a gesture is detected. As an example of the function associated with this gesture, the movement of the finger can rotate some or all of the content displayed on the device.

[0162] In certain embodiments, the gesture may include a movement of the wearable device, for example, made by the arm wearing the device. The movement can be detected by any suitable sensor, such as an inertial sensor, an orientation sensor, or any suitable combination thereof. Figures 59-66 illustrate example gestures that involve detecting a gravity vector relative to the device (eg, pointing in the direction of the front of the device or pointing down through the substrate) and detecting subsequent movement of the device relative to the gravity vector. For example, Figure 59 may include detecting that gravity is pointing downward through the front in step 5905, and detecting that the device is pointing along the gravity vector in step 5910.

The acceleration of that axis is detected in step 5915 that the acceleration of the device is maintained for a certain time step, and the gesture is detected in step 5920. The gesture in Figure 60 is basically similar to that in Figure 59, except that in step 6005, the gravity vector points downward through the base (instead of the front). Figure 61 illustrates a gesture that uses a gravity vector to determine the orientation/position of the device, for example, the device is not next to the user's body. The movement of the device from the detected orientation (for example, perpendicular to the gravity vector) can be detected, resulting in a gesture. For example, the detected gravity orientation may indicate that the arm is not on the side of the body in step 6105, the lateral acceleration of the device may be detected in step 6110, the acceleration may be detected in a period of time in step 6115, and in step 6120 Gestures can be detected in. As indicated in Figures 59-61, detecting one aspect of motion (for example, the duration of acceleration) can trigger a gesture, and each range of one aspect (range of the duration of motion) may each correspond to a different gesture. Figures 62-63 illustrate the rotational movement of the device. As in Figure 61, the detection of the initial orientation or position of the device can be part of the gesture detection. For example, the gesture of FIG. 62 may include detecting that the gravity vector indicates that the arm is not to the side of the body in step 6205, and detecting some kind of rotational movement in step 6210, and in step 6215 It is estimated that the radius of the rotational movement is large enough for the elbow movement, the relative rotation is estimated in step 6220, and the gesture is detected in step 6225. As another example, the gesture of FIG. 63 may include detecting that the gravity vector indicates that the arm is not on the side of the body in step 6305, detecting a certain rotational movement in step 6310, and estimating the radius of the rotational movement for the wrist in step 6315. The motion is small enough, the relative rotation is estimated in step 6320, and the gesture is detected in step 6325. As shown in Figures 62-63, the gesture may include estimating the type of rotation of the device, such as rotation mainly from the shoulder (Figure 62), rotation mainly from the elbow (Figure 63), or any other suitable rotation. In addition to or instead of the radius of rotation, the gesture may also include detecting the amount of rotation, the duration of the rotation, the radial acceleration of the rotation, any other suitable aspect of the rotation, or any suitable combination of the foregoing.

[0163] As with FIGS. 61-63, FIG. 64 indicates that a gesture involves the initial orientation or position of the detection device. For example, the gesture of FIG. 64 may include detecting that the gravity vector indicates that the arm is not to the side of the body in step 6405, detecting the lateral acceleration of the arm along the axis of the arm in step 6410, and detecting that the acceleration remains in step 6415. For a period of time, and in step 6420, a gesture is detected. Figure 65 illustrates that a gesture may include movement of the device along the axis of the appendage that is wearing the device, such as the acceleration of the device along that axis. The gesture may include an impact along the path of movement (for example, caused by a hand stopping or touching an object) and subsequent reversal of the movement. The back and forth movement can be repeated until the movement stops or the hand returns to a certain position, such as the user's side. In certain embodiments, different gestures may be based on the number or frequency of back and forth movements. For example, the gesture of FIG. 65 may include detecting that the gravity vector indicates that the arm is not on the side of the body in step 6505, detecting that the hand is in motion in step 6510, and detecting impulse (impact) along the path of movement in step 6515. ), in step 6520, the reverse movement of the hand along the same linear path is detected, steps 6515 and 6520 are repeated as appropriate, in step 6525, it is detected that the movement has stopped for a period of time, and the gesture is detected in step 6530.

[0164] FIGS. 66-68 illustrate example gestures based on detection of a motion matching a predetermined motion template, which may be user-customizable or user-created. In certain embodiments, the customizable gestures may include the initial position or orientation of the device, movement in a specific direction or some aspect of the movement, the stop and start of the movement, the duration of the movement, or any other suitable movement parameters. In certain embodiments, some or all of these parameters may be user-customizable. In certain embodiments, the detected gesture may be determined by matching the detected motion to the most recently available motion template. For example, as shown in Figures 66-68, a gesture may correspond to the horizontal position or movement of an arm or finger. For example, as shown in FIG. 66, the gesture may include detecting a gravity vector downward passing through the bottom of the base of the device in step 6605, detecting forward and inward movement in step 6610, and matching in step 6615 A motion template (for example, using the heuristic, numerical, or pattern-based gesture recognition module of FIG. 19), and a gesture is detected in step 6620. Figure 67 can include

In step 6705, the gravity vector passing sideways through the bottom of the base of the device is detected. In step 6710, the forward and inward motions are detected. In step 6715, the motion template is matched (for example, using the probe of FIG. 19, Numerical value or style-based gesture recognition module), and a gesture is detected in step 6720. Figure 68 may include detecting that the gravity vector indicates that the arm is not on the side of the body in step 6805, detecting the movement of the device in step 6810, detecting the movement stop in step 6815, and matching the motion template in step 6820, in step 6825 The best motion template matching is selected in, and the gesture is detected in step 6830. Although FIGS. 66-68 illustrate specific examples of customizable gestures corresponding to specific motion templates, the present disclosure contemplates any suitable gestures (or any aspect thereof) that can be customized by the user of the device and detected by any suitable sensor. ).

[0165] In certain embodiments, the gesture may optionally include detecting a certain non-motion or non-orientation input. For example, Figures 69-71 illustrate gestures that include detection of sound, although the illustrated gestures do not require such detection. Figure 69 illustrates sound output (for example, ringing from an incoming or outgoing phone call) or response, followed by some movement of the device (for example, the device is placed in front of the user). For example, an audio response or output is initiated in step 6905, upward movement is detected in step 6910, the stop of upward movement is detected in step 6915, the gravity vector is within a predetermined window in step 6920, and it is detected in step 6925 gesture. In certain embodiments, the gesture may include the detection of a gravity vector in a certain orientation or orientation window, as shown. The gesture of FIG. 69 may also include detecting the position of the user's hand/finger. As an example of a function that can be associated with the gesture shown in FIG. 69, if a finger is placed near the ear or face at the indicated position, the user can answer or make a phone call. Figure 70 and steps 7005-7025 illustrate example gestures that have similar attributes to those described for Figure 69, but involve different orientations of the user's hand/finger. Figure 71 illustrates an example gesture including sounds generated by the user (for example, generated by the user snapping their fingers), these sounds are detected by the microphone associated with the device Measured. For example, Figure 71 may include detecting that the gravity vector indicates that the arm is not to the side of the body in step 7105, detecting a movement with a relatively high acceleration in step 7110, and detecting a sudden change of one or more sound frequencies in step 7115. Changes, and a gesture is detected in step 7120. As shown in FIG. 71, snapping motion can be detected only by motion generated by snapping fingers alone (for example, by the vibration of the user's hand/skin or by a certain degree of rotation or the rate of change of rotation caused by snapping fingers), or It can be detected by a combination of motion plus auditory input generated by snapping. In certain embodiments, to detect a gesture, an auditory confirmation must be detected within a predetermined time of the movement.

[0166] FIGS. 72-73 illustrate example gestures involving periodic movement of the device, such as shaking the arm on which the device is located in a horizontal or vertical direction. Figure 72 illustrates a gesture including detecting that the gravity vector indicates that the arm is not beside the body in step 7205, detecting that the device is moving forward on the axis in step 7210, and detecting that the device is moving on the same axis in step 7215. After moving, repeat steps 7210 and 7215 as needed, and a gesture is detected in step 7220. Figure 73 illustrates a gesture including the detection of the gravity vector indicating that the arm is not beside the body in step 7305, the detection of vertical movement of the device on the axis in step 7310, and the detection of the device moving on the same axis in step 7315 After moving, repeat steps 7310 and 7315 as needed, and a gesture is detected in step 7320. Figure 74 illustrates an example gesture involving adjustment of the position/orientation of the device relative to the user's body. For example, the gesture of FIG. 74 may include detecting that the gravity vector indicates that the arm is beside the body in step 7405, detecting that the gravity vector indicates that the arm is not beside the body in step 7410, and detecting the gesture in step 7415. Any suitable function can be associated with the gestures of Figures 72-75, such as waking the device from a low power state. Figure 75 illustrates an example gesture involving the height of the device or the opening of the slave device. The relative change in the height of the equipment from start to stop. In addition to the height of the device, the gesture may also include the orientation of the device before, during, or after the gesture. For example, the gesture may include detecting that the gravity vector indicates that the arm is not beside the body in step 7505, detecting upward movement in step 7510, detecting the stop of upward movement in step 7515,

In step 7520, it is detected that the gravity vector points to the side of the base passing through the device, and in step 7525, the gesture is detected. Any appropriate function may be associated with the gesture of Figure 75, such as activating equipment paired with the device, turning on one or more lights in the room, or activating equipment near the device.

[0167] In certain embodiments, gestures may include direct interaction with the body or strap of the wearable device. For example, Figure 76 illustrates a gesture involving contact with a touch sensitive area of a strap worn on the user's wrist. This gesture may include detecting that the device is not in the locked state in step 7605, detecting that there is no touch on the strap in step 7610, detecting a touch on the strap in step 7615, and decoding the touch position in step 7620 , And a gesture is detected in step 7625. FIG. 77 illustrates that touches at multiple locations may be determined as a single gesture, such as unlocking the device or some aspects of the device. This gesture may include detecting that the device is not in the locked state in step 7705, detecting that there is no touch on the strap in step 7710, detecting a touch on the strap in step 7715, and decoding the touch position in step 7720 , The action is decoded in step 7725, and the gesture is detected in step 7730. Figure 78 illustrates that a gesture may include contacting a touch sensitive area of the device and sliding across the touch sensitive area while maintaining contact with the device. This gesture may include detecting that the device is not in the locked state in step 7805, detecting that there is no touch on the strap in step 7810, detecting a touch on the strap in step 7815, and detecting (one or Multiple) the movement of the touch point, in the step The relative motion is decoded in 7825, and a gesture is detected in step 7830. In certain embodiments, the gesture may include the duration of the contact, the physical area of the contact (eg, with one finger or two fingers), the sequence of the contact, the pressure generated by the contact, or any other suitable contact-related attributes. Although FIGS. 76-78 illustrate contact with a touch-sensitive area on the strap, the present disclosure contemplates that gestures may involve contact on a touch-sensitive area at any suitable location of the device, such as the device strap , Ring, display, or any suitable combination thereof. For example, Figures 79-80 illustrate contact with a touch sensitive area on the ring of the device, similar to the gestures of Figures 77-78. For example, the gesture may include detecting that the device is not in the locked state in step 7905, detecting that there is no touch on the ring in step 7915, detecting a touch on the ring in step 7920, and detecting the gesture in step 7925. As another example, the gesture may include detecting that the device is not in a locked state in step 8005, detecting that there is no touch on the ring in step 8010, detecting a touch on the ring in step 8015, and detecting a touch in step 8020. The movement of the point, the relative movement is decoded in step 8025, and the gesture is detected in step 8030. Figure 81 illustrates a gesture involving multi-touch with the touch-sensitive area on the front of the device Contact, and detect subsequent movement of the contact point caused by, for example, the movement of the finger in contact with the touch-sensitive area or the movement of the wrist/hand wearing the device. The gesture may include detecting that the device is not in the locked state in step 8105, detecting that there is no touch on the surface in step 8110, detecting that at least two fingers touch the surface in step 8115, and detecting the touch point in step 8120 The relative movement is decoded in step 8125, and the gesture is detected in step 8130. The movement of the wrist/hand can be detected by, for example, an inertial sensor in the device, thereby allowing different ways of moving the touch point into two different gestures. Figure 82 illustrates that a gesture involves an initial contact with the device, which can be detected by one or more proximity sensors on or in the device or inertial sensors on or near the device. The gesture may involve detecting the continuation of the contact, indicating, for example, that the user has put on the device. For example, the gesture may include detecting that there is no contact with the rear or belt proximity sensor in step 8205, detecting the contact of the proximity sensor in step 8210, detecting that the contact continues in step 8215, and detecting in step 8220 gesture. The gesture of Figure 82 can unlock or power up a sleeping device, or provide any other suitable function.

[0168] In certain embodiments, the gesture may include contact with the skin near the device. Figure 83 illustrates a gesture involving tapping on the skin near where the device is worn. The tap can be detected by the vibration sensor in the device. The tapping motion can be confirmed by, for example, one or more acoustic sensors detecting the sound generated by the tapping gesture. For example, the gesture can include

Including detecting that the device is unlocked in step 8305, detecting a movement with relatively high acceleration in step 8310, detecting a sound such as tapping in step 8315, and matching the movement or sound to a pattern in step 8320 , And a gesture is detected in step 8325. FIG. 84 illustrates a gesture involving a swipe of the skin near the device, which can be detected and confirmed by the sensor described in FIG. 83 above. For example, the gesture may include detecting that the device is unlocked in step 8405, detecting a movement with a relatively high acceleration in step 8410, detecting a sound such as tapping in step 8415, and detecting on the skin in step 8420. The vibration or sound of the lateral movement is matched to a pattern in step 8425, and a gesture is detected in step 8430.

[0169] In certain embodiments, gestures may involve detecting symbolic gestures made by hands that are not wearing the device. E.g, Such gestures can be detected, for example, by any suitable forward sensor on or near the display of the device, the sensor being oriented so that the hand of the unworn device is in the viewing angle of the sensor. Fig. 85 illustrates an example gesture involving a forward sensor detecting movement of multiple fingers, such as tapping of a finger. For example, the gesture may include determining that the device is in a predetermined orientation in step 8505, detecting the fingertip in step 8510, detecting the movement of the fingertip in step 8515, or detecting a tapping sound in step 8520, and in step 8520 One or more gestures were detected in and 8530. Figure 86 illustrates an example gesture involving the movement of a single finger. For example, the gesture may include determining that the device is in a predetermined orientation in step 8605, detecting the fingertip in step 8610, detecting the movement of the fingertip in step 8615, or detecting tapping sound in step 8525, and in step 8620 One or more gestures were detected in. Figure 87 illustrates that a gesture involves detecting the movement of the hand holding the object, detecting the movement of the object, locking onto the object, and then detecting the subsequent movement of the object. As a specific example, the gesture may include detecting that the device is in a predetermined orientation in step 8705, detecting the hand in step 8710, detecting the movement of the hand in step 8715, and detecting additional objects to move with the hand in step 8720, exist Lock on the object in step 8725, detect the movement of the object in step 8730, and detect the gesture in step 8735. For example, the object may be a pen or other stylus-like instrument, and the forward sensor on the device may detect the writing movement of the instrument to generate/store text on the device or another device communicating with the wearable device, for example. The example of FIG. 87 may allow the user to generate drawings, annotations, or other written content without actually generating the written content on the display or other writing surface. As will be described more fully in this article, any appropriate gesture or combination of gestures can be used to influence or initiate augmented-reality (AR") functions, and can be used to perform tasks using AR functions. For example, Figures 85-87 The gestures can be used to capture user interactions with a virtual keyboard, virtual mouse, or virtual touch screen and these interactions can generate input on a wearable device or any other suitable paired device. Although the present disclosure describes symbolic gestures and object detection (and associated Functions), but the present disclosure contemplates any suitable symbolic gestures, detection of any suitable objects, and such gestures associated with any suitable functions.

[0170] In certain embodiments, the gesture may involve the entire appendage of the fixed or wearable device. For example, Figures 88-92 illustrate example gestures involving movement of the arm wearing the device. Gestures may include detecting the initial position of the arm (for example, via an accelerometer that detects the direction of the gravity vector), detecting the movement of the device (via the arm), detecting the corresponding change in the gravity vector, and detecting that the arm has stopped moving. Such gestures can also include detecting the duration of the movement, the amount of movement (for example, detecting a large movement radius, confirming that the entire arm is moving), the acceleration of the movement, or any other appropriate movement-related attributes. As shown in Figures 88-92, gestures can involve detecting arm movements above the head, to the front, to the side, to the back, or down from an initially higher starting position. For example, the gesture may include detecting the gravity vector indicating that the hand is on the side of the body in step 8805, detecting the upward movement of the hand in step 8810, detecting the gravity vector indicating that the hand is above the head in step 8815, and detecting in step 8820 When the hand stops moving, and a gesture is detected in step 8825. As another example, the gesture may include detecting the gravity vector in step 8905 to indicate that the hand is on the side of the body, and detecting the hand in step 8910.

In step 8915, the gravity vector indicates that the hand is horizontal, the hand stops moving in step 8920, and the gesture is detected in step 8925. As another example, the gesture may include detecting that the gravity vector indicates that the hand is horizontal in step 9005, detecting that the hand moves downward and backward in step 9010, and detecting that the gravity vector indicates that the hand is on the side in step 9015. It is detected in step 9020 that the hand stops moving, and in step 9025 a gesture is detected. As another example, the gesture may include detecting the gravity vector indicating that the hand is on the side of the body in step 9105, detecting the upward and backward movement of the hand in step 9110, and detecting the gravity vector indicating that the hand is horizontal in step 9115. It is detected in step 9120 that the hand stops moving, and in step 9125 a gesture is detected. As another example, the gesture may include detecting that the gravity vector indicates that the hand is on the side of the body in step 9205, detecting the upward and outward movement of the hand in step 9210, and detecting that the gravity vector indicates that the hand is horizontal in step 9215. In step 9220, it is detected that the hand stops moving, and in step 9225, a gesture is detected. In certain embodiments, the gesture may involve the movement of the entire body and not just the appendages wearing the device.

[0171] In certain embodiments, the user can interact with the device via various input mechanisms or types. These input mechanisms or types include, for example, an outer loop, a touch-sensitive interface (such as a touch-sensitive layer), and gestures performed by the user (herein Description) or voice interface (for example, including voice input and voice recognition, for applications including text input, communication, or search). Furthermore, in certain embodiments, the user can interact with the graphical user interface presented on the circular display of the device via any of these input mechanisms or types.

[0172] The user of the wearable electronic device can use the outer ring to interact with the device (including, for example, a graphical user interface presented on a circular display). In certain embodiments, the outer ring may be touch sensitive, so that a user's touch on one or more parts of the ring can be detected as input to the device and interpreted, causing the device to take one or more actions (eg Within the graphical user interface of the device). As an example, the touch-sensitive outer ring may be a capacitive ring or an inductive ring, and the user of the device may perform any appropriate touch gestures on the touch-sensitive ring to provide input to the device. The input may include, for example, using one hand to command sweeping the ring, using two or more hands to command sweeping the ring, using one or more fingers to perform a rotation gesture, or squeezing the ring. In certain embodiments, the outer ring can be rotatable, so that the physical rotation of the ring can serve as an input to the device. Furthermore, in certain embodiments, the outer ring can be clicked (for example pressed down) or squeezed. Any embodiments of the outer ring can be appropriately combined so that the ring can be one or more of touch sensitive, rotatable, clickable (or pressable), or squeezable. Different forms of input from the outer ring (for example, touch, rotate, click or press or squeeze) may be interpreted differently depending on, for example, a combination of the forms of input provided by the user. As an example, the rotation of the outer ring may indicate an input other than the rotation combined with a click or press on the ring. In addition, when the user provides input via the outer loop, Provide feedback to the user, including tactile feedback, audio feedback, or visual feedback as described in this article.

[0173] FIG. 93A illustrates an example of a user clicking (for example, pressing down) on the outer ring, indicated by arrow 9310. FIG. 93B illustrates an example of the user squeezing the outer ring, indicated by arrow 9320. FIG. 94A illustrates an example in which the user rotates the outer ring so that the content 9410 of the graphical user interface of the device changes according to the rotation (for example, to the right). FIG. 94B illustrates an example where the user performs a rotation gesture on the touch-sensitive outer ring, and the ring itself does not rotate, so that the content 9420 of the graphical user interface of the device changes according to the rotation (for example, to the right). FIG. 94C illustrates an example where the user rotates the outer ring and simultaneously presses or clicks the ring, so that the content 9430 of the graphical user interface of the device changes according to the rotation (for example, to the right) and the pressing or clicking.

[0174] In a specific embodiment, the touch-sensitive interface of the device (such as the touch-sensitive layer) can accept user touch input and allow the device to determine the xy coordinates of the user's touch, and identify multiple points of touch contact (such as the touch-sensitive layer). At different areas) and distinguish between different lengths of touch interaction (for example, the distinction includes swiping, single tap or

Gesture including double tapping). Touch gestures (described herein) can include swipe or drag in multiple directions, pinch, double tap, press or push on the display (which can cause physical movement of the display in the upward or downward direction), long press, multiple Point touch (such as using multiple fingers or instruments to touch or gesture anywhere on the touch-sensitive interface) or rotary touch gestures. Figure 95A illustrates an example of a user tapping 9510 on a touch-sensitive interface (eg, a touch-sensitive layer) to provide input to the device. The precise xy coordinates of the user's tap can be determined by the device through input from a touch-sensitive interface (such as a touch-sensitive layer). FIG. 95B illustrates an example in which the user performs a clockwise rotation gesture 9515, a counterclockwise rotation gesture 9520, a vertical swipe gesture 9525, and a horizontal swipe gesture 9530, respectively. FIG. 95C illustrates that the user simultaneously uses one, two, or three contact points 9535 (for example, using one, two, or three fingers or instruments) to touch the display (including a touch-sensitive layer with multi-touch sensing capabilities). Example. FIG. 95D illustrates an example in which a user performs a touch gesture having multiple contact points with a touch-sensitive interface. In this example, the user can use two fingers to perform the spread gesture 9540, the pinch gesture 9545, the clockwise rotation gesture 9550, or the counterclockwise rotation gesture 9555.

[0175] In certain embodiments, the graphical user interface of the device may operate according to interaction and transition models. The model can, for example, determine how the modes including applications, functions, sub-modes, confirmations, content, controls, active icons, actions, or other features or elements can be organized (eg, hierarchically) within the graphical user interface of the device.

[0176] In one embodiment, a graphical user interface (GUI) includes multiple top-level screens, each of which corresponds to a different mode or application (or sub-mode, function, confirmation, content, or any Other features). Each of these applications can be at the same level of the hierarchical system of GUI interaction and transformation models. FIG. 96A illustrates an example layout of the hierarchical system within the GUI, where a plurality of top-level screens 9602-9606 and 9610-9614 each correspond to a different application, and one of the top-level screens 9608 (main screen) corresponds to a clock. The state transition within the GUI may be an event triggered by input from an input source such as the user of the device. Input from the user of the device or from another input source (for example, via any of various input mechanisms or types, including outer loops, touch-sensitive interfaces, gestures, voice, or sensors) can cause transitions within the GUI (for example, from One top-level screen shifts to another). For example, the input may cause the GUI to transition from the home screen 9608 (such as a clock) to an application (such as 3 or 4) or from one application to another application. If the user rotates the outer ring to the right, for example, the GUI can transition from the home screen 9608 to the application 49610, and if the user rotates the outer ring to the left, the GUI can transition from the home screen 9608 to the application 39606. In other real In an embodiment, the context (for example, determined by a sensor or other input source on the device) can cause the GUI to transition from the home screen to the application or from one application to another application.

[0177] In one embodiment, the model may include operability regarding the distinction between "left" and "right" sides with respect to the home screen. As an example, one or more of the top-level screens can interact with the GUI and a fixed (for example, always available to the user) or contextual or dynamic (for example, available depending on the context) mode in the hierarchical system of the model. Or application (or other characteristics) associated. Contextual screens can, for example, reflect the users most recently used mode, application or function, users recently added (for example, downloaded) mode, application or function, temporarily registered device (which can, for example, enter or leave the communication range of the device when the device is used) ), a mode, application or function that is a user's "favorite" (for example, explicitly designated by the user) or a mode, application or function recommended for the user (for example, based on the user's previous activities or current situation). FIG. 96B illustrates an example layout of the hierarchical system within the GUI, where contextual or dynamic applications 9616-9620 and fixed applications 9624-9628 are separately grouped, and the left side (relative to the main clock screen 9622) includes contextual applications, and The right side includes stationary applications. As an example, the dynamic application 019620 may be the most recently used application, and the dynamic application 029618 may be the second most recently used application, and so on.

[0178] In a specific embodiment, the top level of the hierarchical system of the GUI interaction and transition model may only include "face", and

The next level of the hierarchy can include applications (or any other characteristics). As an example, the top level of the hierarchy can include a home screen (such as a clock) and one or more faces, each of which corresponds to a different type of background, mode or activity, such as wallpaper (for example, customizable by the user), weather information, calendar Or daily activity information. Each face can show the time in addition to any other information displayed. In addition, the currently displayed face may be selected by the user (for example, via any appropriate input mechanism or type) or automatically changed based on the context (for example, the user's activity). The face on the left side of the home screen can be contextual, and the face on the right side of the home screen can be fixed. Figure 97 illustrates an example layout of the hierarchical system in the GUI, where the top level of the hierarchical system includes faces 9710-9770 (including clock face 9740) and the next level of the hierarchical system includes applications 9715-9775.

[0179] In certain embodiments, input from a user of the device or input from another input source (for example, via any of various input mechanisms or types, including outer loops, touch-sensitive interfaces, gestures, voice, or Sensors) or the use context of the device can cause a transition within the GUI from a screen at one level of the hierarchical system of the GUI's interaction and transformation model to a screen at another level of the hierarchical system. For example, a selection event or input made by the user (such as touch or tap on the display, voice input, eye gaze, click or press on the outer ring, squeeze on the outer ring, any appropriate gestures, internal muscles detected by the sensor Motion or other sensor input) can cause the transition from the top-level screen in the GUI to a deeper nested screen in the hierarchy. If for example the current screen is the top-level screen associated with the application, Then the selection event (such as pressing the ring) selects the application and causes the GUI to transition to a deeper nested screen. This second screen may, for example, allow interaction with features of the selected application and in certain embodiments may correspond to the main function of the selected application. There may be multiple screens at this second nesting level, and each of these screens may correspond to different functions or features of the selected application. Similarly, a "return" selection input or event made by the user (such as a double press on the outer ring or a touch gesture in a specific part of the display) can cause the GUI to go from one screen (such as a feature of a specific application) to a hierarchical system. The transition of another screen at a higher level (for example, the top-level application screen).

[0180] FIG. 98A illustrates an example of the function or mode 9805 of a specific application of the device and the use of the function or the operation of the interaction and transition model of the application 9810. As an example, if the application is a camera, the functions, modes, or other elements of the camera application may include a picture mode, a video mode (for example, with live view), and turning on or off the flash. Various functions, modes, or other elements can be accessed through transformations within a single layer of the model hierarchy. These intra-layer transitions can occur when receiving or determining a specific type of transition event or input from an input source such as the user of the device (for example, counterclockwise or clockwise rotation of the outer ring) or when determining the specific use situation of the device . In certain embodiments, the transition event input may also include, for example, touch or tap on the display, voice input, eye gaze, click or press on the outer ring, squeeze on the outer ring, any appropriate gesture, detected by the sensor Internal muscle movement or other sensor input. In order to select and use the functions, modes, or other elements of the application, the user can provide a specific type of selection event or input (for example, tap or touch on the display, press or click on the outer ring, specific gestures or sensor input) to cause the GUI The deeper inter-layer transition from the inner to the hierarchical system. As an example, in order to shoot a video, the user may tap the screen associated with the video mode feature of the camera application. Once the video is taken one by one at this deeper level of the hierarchy, the user will The GUI can be made to switch between different options in this layer, if these options are available (for example, options related to the video mode). In a specific embodiment, the user can select one of the options in a deeper layer, so that the GUI transitions to a deeper layer. As an example, once the video is recorded in the video mode, the user can tap the display again to transition the GUI to a deeper layer, in which case the deeper layer may include an option to stop recording the video. In addition, users can return to higher levels of the hierarchy by providing specific types of selection events or inputs (such as the "return" input described in this article). As an example, once the video is recorded in the video mode, the user can touch a specific "return" part of the display so that the video recording is taken

Cancel and cause the GUI to transition to the screen associated with the video mode feature of the camera application (for example, in the feature layer of the hierarchical system). The GUI interaction and transformation model hierarchy can have any number of layers and any number of elements (for example, functions or content) within a single layer. FIG. 98B illustrates an example of the operation of the interaction and transition model regarding the content 9815 on the device. In this example model, the behavior of the content can be similar to the application, except that if the user selects the content 9815 (such as a photo) and the GUI transitions to a deeper level in the hierarchy, the first option in the menu of options related to the content 9820 can be shown (for example, options like delete photos or share photos). FIG. 98C illustrates an example of the interaction and the operation of the transition model regarding the control 9825 on the device. The function of the control element can be similar to a knob in that it can modify the value over a certain range of possible values. User input to the device (eg, rotating the outer ring to the right or left) can modify the value or state 9830 associated with the control element 9825. The value modified by the control element may be substantially continuous in nature (for example, the zoom level of a camera, or the volume level of a TV) or may be substantially discrete in nature (for example, a TV channel). In certain embodiments, where the value modified by the control is discrete in nature, a certain user input (for example, pressing the outer ring) may "submit" the selection of the value. Figure 98D illustrates the application 9835 and the main application on the device An example of the interaction of the function 9840 and the operation of the transition model. As an example, each mode or function of the device (such as a camera or augmented reality function) may be an application on the device. Transitions within a single layer (for example, performed upon receiving specific user input such as the rotation of the outer ring) allow the user to change the application, mode, or function of the device. The transition between layers (for example, when a specific user input such as a tap on the display is received) allows the user to enter the deeper layers of the hierarchy associated with the selected application, mode, or function (or leave more Deep).

[0181] FIG. 98E illustrates an example of the operation of the interaction and transition model with respect to an action 9845 on the device (for example, within an application). As an example, in the camera application, the captured image can be selected, and there can be one or more actions available for the selected image, such as deleting the image, sharing the image on Facebook, sharing the image on TWITTER, or sending Email with this image. In this example, GUI transitions within the "actions" layer (performed for example when a specific user input such as the rotation of the outer ring is received) allows the user to see the different actions to be taken. The transition between layers (performed for example when a specific user input such as a tap on the display is received) allows the user to enter a deeper layer of the hierarchy associated with the selected action (or leave a deeper layer). In this example, the deeper level entered by selecting action 9845 shows secondary information 9850 or confirmation (eg, confirmation that the application is sending image information to the selected sharing service). Confirmation 9855 (for example confirming that the image has been sent) can also be shown in this deeper level. The GUI can automatically transition back to a higher layer (such as the action layer). However, there can be a deeper layer of the hierarchical system that includes confirmation information, and the GUI can enter this deeper layer when the user enters or automatically. FIG. 98F illustrates related icons (e.g., including An example of the interaction of the active icon 9860 of the level on/off option and the switching of the state of the icon 9865 and the operation of the transition model. As an example, the TV paired with the device can be indicated by an active icon, such as a TV screen. In this example, GUI transitions within the top level of the device/application (for example performed upon receiving specific user input such as the rotation of the outer ring) allow the user to view different applications, devices, or other features. The TV may appear in the menu in the device's GUI even when the TV is turned off, but the TV must be turned on before it can be used. If the user selects the TV when the TV is turned off (for example, by tapping on the display when the TV icon is displayed by the GUI) 9860, the GUI can transition to a state in the deeper layers of the interaction and transition model hierarchy, in which the television is Open 9865_OWhen the TV is turned on, the icon associated with the TV (displayed in the top layer of the model in the GUI, for example) 9870 can be changed to 9875 directly indicating that the TV has been turned on, as shown in FIG. 98G. If the user selects the TV again (now it is turned on), the GUI can transition to a deeper layer of the hierarchy, where the TV's functions or capabilities (such as volume or channel changes) are exposed. In a specific embodiment, the option to turn off the TV again may be the first menu item in this deeper layer of the hierarchy to

Enable quick access to the shutdown function (for example, if the user accidentally turns on the TV). In a specific embodiment, if the user selects a TV when the TV is turned off, the TV can be turned on and the icon associated with the TV can be changed to directly indicate that the TV has been turned on, and the GUI does not change to a different layer of the hierarchy or different User interface. The active TV icon can therefore directly indicate the status of the paired TV in the top level of the hierarchy (such as the main menu).

[0182] FIG. 99 illustrates an example of the interaction and transition model hierarchy regarding the GUI of the image capturing application. In this example, the first screen 9902 that arrives after selecting an application (at screen 9900) may correspond to the "live view" function of the application. Other fixed features of the image capture application, including video mode 9904, zoom 9906, or flash 9908, are available on the right side of the main function screen 9902 on the home page of the selected application. The dynamically or contextually available features of the selected application (eg, captured image 9910) are available on the left side of the main function screen on the home page. The selection event at this functional level of the hierarchical system can cause a transition in the GUI to another nested level deeper in the hierarchical system. For example, if the user selects the "zoom" function, the GUI can transition to screen 9912, where the user can use any appropriate input to control the camera's zoom settings (for example, rotating the outer ring to the right to increase the zoom or rotating the outer ring to the left Ring to reduce the zoom). Similarly, the user may be able to control the status of different features (such as turning on or off the flash feature 9914, or switching from the picture mode to the video mode 9916), browse content (such as 9918-9922), and enter the deeper layers of the hierarchy, where you can Take action 9924-9930, or enter another deeper layer of the hierarchy, where once you choose an action, you will provide confirmation 9932-9938o

[0183] In certain embodiments, the interaction layout may construct an interaction and transition model of the GUI of the device. The interaction layout can be applied to any appropriate interaction model and does not need to rely on, for example, any specific type of motion or animation within the GUI of the device. Although specific examples of interaction layouts are discussed below, any appropriate interaction layout can be used to construct interaction and transformation models.

[0184] As an example, a translational linear interactive layout can construct an interaction and transition model of the GUI of the device. In the translational linear GUI, the elements or features in the layer can be arranged on the left and right of the currently displayed element or feature. User input such as rotating the outer ring in a clockwise or counterclockwise direction navigates within a single layer of the model hierarchy. As an example, rotate the outer ring clockwise by one rotation increment to display the element or feature on the right (such as the next element), and rotate the outer ring by one rotation increment to display the element or feature on the left (such as the previous element) ο In certain embodiments, a fast rotation clockwise or counterclockwise can cause the GUI to perform accelerated browsing. In such an embodiment, a single revolution may cause the GUI to transition through multiple elements or features, rather than a single element or feature, as described herein. Different user inputs can navigate between layers in the model hierarchy (for example, deeper or higher). As an example, if the user touches or taps the touch-sensitive layer of the display, the GUI can transition to a deeper level in the model hierarchy (for example, confirm the user's selection or provide options related to the selection). Instead of or in addition to touch or tap based input, any appropriate input made by the user can cause the GUI to transition between layers in the model hierarchy.

[0185] As another example, if the user presses a specific area of the touch-sensitive layer of the display (for example, designated as a "back" button), or if the user double taps the touch-sensitive layer of the display, the GUI may transition to a model hierarchy A higher level in the middle (for example, transition to the previous level). For example, if the user performs a long press on the display or screen, the GUI can transition back to the main screen (such as a clock). Without additional user input, the GUI can also transition back to the main screen after a predetermined period of time (such as a timeout period). As described herein, as the user starts to rotate the outer ring in a clockwise or counterclockwise manner, for example, the GUI transitions within the same layer, and the next user interface element or feature on the right or left side (such as breadcrumbs in the same layer) Icons) can start to appear separately, and current user interface elements or features can start to disappear.

[0186] FIG. 100A illustrates an example of a translational linear interactive layout. In this example, the GUI elements 10001, 10002,

10003 and 10004 are in the same layer of the interaction and transformation model hierarchy of the translation linear GUI. GUI elements 10002A, 10002B and 10002C are elements in the deeper second layer of the hierarchy and are child elements of element 10002. As an example, the first layer may include devices that are paired with devices-an element 10001 may represent a car, an element 10002 may represent a TV, an element 10003 may represent a mobile phone, and an element 10004 may represent a home thermostat. Element 10002A can be a volume control element of a TV, element 10002B can be a channel control element of a TV, and element 10002C can be a picture control element of a TV. As another example, if the user clicks on the ring (for example, presses the ring down once), the GUI can transition to a deeper layer in the hierarchy, and then rotate the ring to translate the sub-elements in the deeper layer. Alternatively, the user can pan the sub-elements in a deeper layer by rotating the ring while pressing the ring down. The device may include switches to select how user input is used to navigate between layers.

[0187] As another example, a translational radial (or a circular translation) interactive layout may construct an interaction and transition model of the GUI of the device. In the panning radial GUI, the elements or features in the layer can be arranged above and below the currently displayed element or feature. User input such as rotating the outer ring in a clockwise or counterclockwise direction navigates between the layers of the model hierarchy. As an example, rotating the outer ring clockwise by an increment can make the GUI transition to a deeper level in the model hierarchy (such as entering the layer of a specific application or confirming the choice of an application), and rotating it by an increment counterclockwise can make the GUI transition Go to a higher level in the model hierarchy (for example, leave the application-specific layer to the previous layer). In certain embodiments, fast rotation clockwise or counterclockwise may cause the GUI to perform accelerated browsing, as described herein. In such an embodiment, a single rotation increment can cause the GUI to transition through multiple layers of the hierarchy, rather than a single layer. Different user inputs can be navigated within a single layer in the model hierarchy. As an example, if the user touches or taps the touch-sensitive layer of the display, the GUI may transition to the next element or feature (for example, the element below the currently displayed element). As another example, if the user presses a specific area of the touch-sensitive layer of the display (e.g. designated as a "back" button), or if the user double taps the touch-sensitive layer of the display, the GUI may transition to the previous element or feature ( For example, the element above the currently displayed element). For example, such as If the user performs a long press on the display or screen, the GUI can transition back to the main screen (such as the clock). Without additional user input, the GUI can also transition back to the main screen after a predetermined period of time (such as a timeout period). As described herein, as the user starts to rotate the outer ring in a clockwise or counterclockwise manner, for example, the GUI transitions to a different layer, and the next user interface element or feature (for example, in a different layer) can start to appear, while the current user interface Elements or features can start to disappear. FIG. 100B illustrates an example of a translational radial interactive layout. In this example, GUI elements 1000M0002, 10003, and 10004 are in the same layer of the interaction and transition model hierarchy of the translational radial GUI. GUI elements 10002A, 10002B, and 10002C are elements in the deeper second level of the hierarchy and are child elements of element 10002. As before, the first layer can include devices that are paired with the device-an element 10001 can represent a car, an element 10002 can represent a TV, an element 10003 can represent a mobile phone, and an element 10004 can represent a home thermostat. Element 10002A can be a volume control element of a TV, element 10002B can be a channel control element of a TV, and element 10002C can be a picture control element of a TV.

[0188] As another example, an accordion-type interaction layout may construct an interaction and transition model of the GUI of the device. In an accordion-type GUI, elements or features of multiple layers may be arranged in a circular list structure. For example, rotating in the first direction (for example, by rotating the outer ring) in the list structure through the screen associated with the last element or feature in that direction (for example, the last fixed application of the device) can cause the GUI to transition to the second The screen associated with the last element or feature in the direction (for example, the contextual application used by the device last time). Continuing to rotate in the first direction may cause the GUI to transition through the screens associated with the contextual application in a "reversed" order (for example, from the oldest used to the most recently used). Similarly, rotating the screen of the contextual application that was used the longest ago in the second direction can cause the GUI to transition to be similar to the last stationary application.

The associated screen, and continuing to rotate in the second direction can cause the GUI to transition through the screen of the fixed application in reverse order (for example, from the last fixed application to the first fixed application adjacent to the main screen). In an accordion-type GUI, the currently displayed element or feature can be "expanded" (for example, if selected by the user) so that its sub-elements or sub-features can become part of a single-level list structure. In certain embodiments, an element or feature (when displayed) that has sub-elements may indicate that it has sub-elements by, for example, the visible edges of the sub-element. User input such as rotating the outer ring in a clockwise or counterclockwise direction navigates within a single layer of the model, which may include elements or features and sub-elements or sub-features of the selected element or feature. As an example, rotating the outer ring clockwise by one increment can display the element or feature on the right (such as the next element), and rotating it counterclockwise by one increment can display the element or feature on the left (such as the previous element). In certain embodiments, a fast rotation clockwise or counterclockwise may cause the GUI to perform accelerated browsing. In such an embodiment, a single rotation increment may cause the GUI to transition through multiple elements or features, rather than a single element or feature. Different user inputs can cause the selection and expansion of elements or features in the model. As an example, if the user touches or taps the touch-sensitive layer of the display, the GUI can expand the features or elements displayed in the existing layer and transition to sub-elements or sub-features. As another example, such as If the user presses a specific area of the touch-sensitive layer of the display (for example, designated as a "back" button), or if the user double-taps the touch-sensitive layer of the display, the GUI can collapse the expanded sub-elements or sub-features and transition to the list Elements or features in. For example, if the user performs a long press on the display or screen, the GUI may transition back to the main screen (such as a clock). Without additional user input, the GUI can also transition back to the main screen after a predetermined period of time (such as a timeout period). As described herein, as the user starts to rotate the outer ring in a clockwise or counterclockwise manner, for example, the GUI transitions within the same layer, and the next user interface element or feature on the right or left (such as breadcrumbs in the same layer) Icons) can start to appear separately, and current user interface elements or features can start to disappear. Figure 100C illustrates an example of an accordion-type interactive layout. In this example, GUI elements 1000M0002, 10003, and 10004 are in the same layer of the interaction and transition model of the accordion GUI. Because element 10002 has been selected by the user, GUI sub-elements 10002A, 10002B and 10002C are expanded and also included in the list structure in the same layer of the model. Thus, the GUI can be changed from the sub-element 10002C to the sub-element 10002B or directly to the element 10003_oHowever, if the user wishes to collapse the child elements (for example by "return" input, such as tapping the screen associated with element 10002 again), the list structure will again only include GUI elements 10001, 10002, 10003, and 10004ο

[0189] In certain embodiments, the GUI may navigate to the home screen based on input received by the user of the device. User input can include, for example, pressing (for example, long pressing) on the touch-sensitive layer, pressing the display, pressing (for example, clicking) on the outer ring, squeezing the outer ring, covering the front of the device (display), covering specific sensors of the device, Rotate the front of the device in the downward direction, Press a soft button (discussed in this article), press a hard button on the device, or shake the device (or any other appropriate gesture). Any one of these inputs or any variation of these inputs (including, for example, shorter durations) can be used as user input for "returning" within the interaction and transition model. Figures 101A-101B illustrate an example of the "return" soft button layout in the GUI. In FIG. 101A, receiving user touch input at the bottom 10110 of the display allows the GUI to confirm the selection or transition to a deeper level in the model hierarchy. Receiving user touch input at the top 10120 of the display causes the GUI to transition "back" or to a higher level in the model hierarchy. Figure 101B illustrates a similar layout, where the "return" area 10130 includes a breadcrumb icon 10135 to indicate to the user where the navigation "return" will transition. In certain embodiments (for example, when the touch sensitive layer is operable to determine the precise xy coordinates of the touch), any area of the display can be designated as a "return" area, a "confirm/select" area, or any other suitable functional area .

[0190] In certain embodiments, the GUI of the device may display certain types of content, including, for example, lists. FIG. 102A illustrates an example of a GUI displaying a vertical list of items. Input from the user (e.g. any appropriate input mechanism or

Type) can cause the selection box 10210 of the GUI to move past the elements of the vertical list. As an example, if the user rotates to the right in a clockwise direction, the selection box 10210 can move from the top of the vertical list to the bottom of the vertical list. Each rotation increment of the outer ring (for example, if the outer ring is moved in discrete increments) will cause the selection box 10210 to move an item in the list. In the example of FIG. 102A, as the user rotates the ring clockwise, the displayed items of the list remain constant, and the selection box 10210 moves downward past the items of the list. In other embodiments, the selection box may remain constant (for example, at the center of the display), and the items of the list may move up or down depending on the direction of rotation of the ring (for example, one item at a time). FIG. 102B illustrates an example of a GUI displaying a horizontal list of items. Input from the user (eg, any suitable input mechanism or type) may cause the selection box 10210 of the GUI to move past the elements of the horizontal list. As an example, if the user rotates to the right in a clockwise direction, the selection box 10210 can move from the left side of the horizontal list to the right side of the horizontal list. Each rotation increment of the outer ring (for example, if the outer ring is moved in discrete increments) will cause the selection box 10210 to move an item in the list. In the example of FIG. 102B, as the user rotates the ring clockwise, the selection box 10210 is displayed on the display The center of the is kept constant, and the items of the list move to the left in response to clockwise rotation (for example, one item at a time). In other embodiments, the display items of the list remain constant, and the selection box moves left or right past the items of the list depending on the rotation direction of the outer ring.

[0191] In certain embodiments, the GUI of the device may display continuous (or substantially connected) content including, for example, graphs or text, vertically or horizontally. In certain embodiments, input from the user (eg, any suitable input mechanism or type) may cause the selection indicator of the GUI to move through the continuous content. In other embodiments, input from the user may cause content to move in and out of the display in the horizontal direction, vertical direction, or any other direction that maps to the user input (and the selection indicator-if present-may remain in a constant position ). In the example of FIG. 102C, a temperature graph is displayed. As the user rotates the outer ring in a clockwise manner, the selection indicator 10220 remains in the center of the display, and the content moves into the display from the right and out of the display to the left. In the example of FIG. 102D, a portion of a larger piece of text 10230 is displayed. As the user rotates the outer ring in a clockwise manner, additional text enters the display from the bottom and leaves the display toward the top. Figures 103A-103D illustrate an example calendar application displayed in the GUI of the device. In FIG. 103A, the user can click or press the outer ring (indicated by arrow 10305) to make the GUI display a circle with options "up", "weekly" (default setting), "monthly" and "daily" Menu 10310ο In Figure 103C, the user can click or press the outer ring again (by Arrow 10305 indicates), confirm the selection of "weekly" and cause the GUI to display a weekly view of the user's calendar 10320. [0192] In certain embodiments, the GUI may display content with a larger size than the display. In such an embodiment, the GUI can zoom or crop (or otherwise reduce or adapt) the content so that all the content can be displayed in the display at the same time. In other embodiments, the GUI does not change the size of the content, but provides the user with the ability to pan through the content one part at a time, for example, using scrolling (described herein).

[0193] In a particular embodiment, the device includes a circular display, and the GUI includes a circular navigation and menu layout. However, the present disclosure contemplates any shape of the display, as well as any appropriate navigation or menu layout of the GUI. The menu layout may, for example, provide the user with a visual indication of where the user is in the GUI interaction and transition model hierarchy. The menu layout may also provide visual indicators that allow the user to distinguish between different types of menu items and show an overall view of the menu options. In addition, the menu can be displayed on any suitable background or content of the device.

[0194] FIG. 104 illustrates an example circular menu layout, where each segment 10410 represents an item or option in the menu and a visual gap such as 10420 separates the options from each other. The default or currently selected item 10430 is on the top of the visual display (but can be anywhere on the display), and can remain on top of the display when the user sets the orientation of the device display in different ways during use. Figures 105A-105B illustrate browsing items in the circular menu

Example of purpose. The user may provide an input such as a clockwise rotation of the outer ring, and in response to this user input, the next item 10520 in the menu (for example, to the right of the currently selected item 10510) may be highlighted for selection. The content 10530 in the center of the display may automatically change to reflect the user's rotary input, or in certain embodiments may only change after the user provides another input (for example, pressing or clicking on the outer ring once the desired menu item is highlighted). FIGS. 105C-105D illustrate an example in which the circular menu is browsed by rotating the outer ring, so that the next item 10550 in the menu (for example, the clockwise or right side of the currently selected item 10540) is highlighted for selection. In this example, the user's input also causes the rotation of the central "pointer" 10560, which points to the highlighted menu segment corresponding to the currently selected menu item. In this example, the content of the center of the display automatically changes to reflect the user's rotation.

[0195] FIGS. 106A-106C each illustrate a different alignment and arrangement of the circular menu layout of the device's GUI. The circular menu may for example be displayed directly on the border of the display (as shown in FIG. 106A) or may be shown on the inner side of the display, or displayed as an overlay on the background of the device (shown in FIGS. 106B-106C) out). Figures 107A-107C illustrate other forms and alignments of the circular menu layout of the device's GUI. As an example, the menu may be composed of line segments 10710 arranged in a circle (of various possible sizes), line segments 10720 arranged in a semicircle, or dots 10730 or 10740 arranged in a circle or a semicircle. In a particular embodiment, the visual indicator of the currently selected or default menu item 10732 can remain at the top center of the display, and the visual indicator of the item 10734 in the menu can be moved to the left or right based on user input (Fig. 107C) ο In other embodiments, the visual indicators of the currently selected or default item 10732 can move past the indicators of the menu items, and the positions of these indicators remain fixed (FIG. 107B) ο In certain embodiments, instead of Line segments or points, visual indicators of items in the menu may be icons associated with the menu items (for example, breadcrumbs icons). Figure 108 illustrates that the menu layout does not need to be circular, but can be any suitable layout, including the index of menu item 10810 The layout of the indicators scattered across the display. Using user input (such as the rotation of the outer ring), different items can be selected according to their position in the menu layout. As an example, if the user rotates in a clockwise manner, the next menu item 10820 in the clockwise direction may be selected.

[0196] FIGS. 109A-109C illustrate different menu layouts of menu items on the "left" and "right" (for example, in the interaction and transition model hierarchy) of the currently selected or displayed menu item 10915. In Figure 109A, all menu items 10910 are evenly distributed on a circular menu surrounding the display. In FIG. 109B, the menu includes a gap that indicates the distinction between the item 10910 on the left and the item on the right of the currently displayed or selected menu item 10915 (for example, according to the interaction and transition model described herein). Figure 109C illustrates an example in which there are more items 10910 on the left side of the currently selected or displayed item 10915 than on the right, so that the size of the left-hand side segment of the circular menu is adjusted to fit the size of the items available for selection. number. In the case of a large number of menu items (for example, exceeding a certain threshold, such as 40 captured images), segments of the circular menu may disappear, and the visual indicator presented to the user may be to allow the user to scroll through various menu items The scroll bar 11020, as shown in FIG. 110A. In other embodiments, a similar scroll bar-type visual indicator 11020 may allow the user of the device to manipulate an absolute or fixed value (eg, camera zoom level) over a fixed value range 11030, as shown in FIG. 110B. In still other embodiments, the length of the scroll bar-type visual indicator can provide the user with Shows the level of a specific value. For example, if the user is using the outer ring of the device to control the volume of the TV, as the user turns the ring (clockwise) to increase the volume level, the visual indicator 11120 will become longer until it surrounds or almost surrounds the entire display , As shown in Figures 111A-111C.

[0197] In certain embodiments, the GUI may display both items of reference or background content and indications of available actions or functions to be performed on the reference or background content. Figure 112 illustrates an example layout of reference content and contextual overlay actions or functions within the GUI. Different types of layouts can be selected based on different types of references or background content presented

(Including those illustrated), for example, to minimize occlusion of reference or background content. For example, if the reference or background content is a picture of a person, you can choose not to obscure the overlay of the center of the photo. In certain embodiments, the perceived brightness of pixels of the reference or background content can be determined on a pixel-by-pixel basis (for example, after overlay). In situations where the contrast between the contextual overlay and the reference or background content (such as an image) is too low (eg based on a predetermined threshold), fuzzy shadows that push the underlying colors in the opposite direction can be used. Example algorithms may include determining the pixels under the overlay, reducing their saturation, taking the inverse of visual brightness (for example, keeping the colors the same but choosing brightness to create contrast), blurring, and creating an underlying reference or between the background content and the overlay Synthesis. Figures 113A-113C illustrate examples 11310-11350 of contextual overlays synthesized with background or reference content (here, images captured by the device's camera). As shown in the figure, contextual coverage can allow users to perform actions or functions (such as deleting images 11130 or sharing images 11325, searching for coffee 11330, searching for restaurants 11340, or making a location a "favorite" location 11350), and provide confirmation to the user (such as Confirm that the image has been shared 11320), or provide any other type of information to the user. In certain embodiments, contextual coverage can be used in the GUI Anywhere in the menu layout, except for the top level of the interaction and transformation model hierarchy.

[0198] In certain embodiments, the icons displayed in the GUI of the device can optimize the energy or battery usage of the device. As an example, the icon may include a mainly black background, and the icon itself is composed of thin white strokes. This may allow the amount of white color on the display to be very low, thereby allowing the energy consumption of the display to be reduced when the GUI is used. The icons displayed in the GUI may also include real-time notifications. For example, the mobile phone icon may include a notification with the number of new voice mails, the email icon may include a notification with the number of new emails, the chat icon may include a notification with the number of new chat messages, and the phone icon may Include notifications with the number of missed calls. In certain embodiments, the GUI of the device only displays colors other than black and white for user-generated content (such as pictures, files, contacts, notifications, or schedules). Other information including menu items can be displayed in black and white.

[0199] In certain embodiments, as the GUI transitions from one element (eg, feature, content, item, or icon) to another (eg, upon receiving input from the user), the GUI may display a visual transition effect. These transition effects may depend, for example, on the type of input received from the user of the device. As an example, a single touch on the display can trigger a specific transition effect, while the rotation of the outer ring can trigger a different set of (possibly overlapping) transition effects.

[0200] In certain embodiments, the user's touch input on the touch-sensitive layer can trigger transition effects including centripetal expansion, directional sliding, and zoom in or zoom out. Figure 114A illustrates a centripetal mode or function expansion or zoom. Figure 114B illustrates the centripetal mode or function retracting or zooming out. Figure 115A illustrates centripetal enlargement of the icon. FIG. 115B illustrates the centripetal reduction of the icon. FIG. 116A illustrates an example of a centripetal icon zooming in with twisting motion. FIG. 116B illustrates an example of zooming out of the centripetal icon with twisting motion. FIG. 117A illustrates an example of centripetal spreading and outward spreading of icons. Fig. Π7B illustrates an example of centripetal folding and inward folding of the icon. FIG. 118A illustrates an example of text sliding vertically into the display, where the text is revealed by unmasking. FIG. 118B illustrates an example in which text slides in horizontally from the left side to the right side of the display. FIG. 118C illustrates an example in which text slides in horizontally from the left side to the right side of the display within a masked area (for example, contextual coverage). FIG. 119A illustrates the horizontal sliding transition of content or icons from right to left. Figure 119B illustrates a horizontal sliding transition from right to left, with a gradual retreat effect; icons or content leaving the screen gradually fade out once they reach the border of the screen, and icons or content entering the screen when they cross the border of the screen Fade in gradually. FIG. 119C illustrates an example of a horizontal sliding transition from right to left with a zoom effect; content or icons that leave the screen are reduced; and content or icons that enter the screen are enlarged to full size.

[0201] In a specific embodiment, the user's rotation of the outer ring may trigger visual transition effects including zoom in/out, directional sliding, blur, masking, page retracting, rotating motion, and acceleration motion. Figure 120A illustrates the low response to the outer ring

Example of acceleration rotation transition. In this example, a single rotation increment can correspond to a single item, so that a counterclockwise rotation (such as one rotation increment) will cause the next element (such as an icon or content item) to enter the screen from left to right, and the elements Scaling does not happen. Figures 120B-120C together illustrate an example of the transition in response to the high acceleration rotation of the outer ring. In this example, a single counterclockwise rotation (such as a single rotation increment) causes the GUI to quickly pan across multiple elements (the size of these elements can be reduced, entering the screen from the left, and leaving the screen from the right) until the user stops turning the ring until. When the user stops turning the outer ring, the element can be enlarged to normal size, and a single icon or content item can fill the display. FIG. 121A illustrates an example of transition within the GUI, where the content is zoomed in in response to the rotation of the outer ring. FIG. 121B illustrates an example of a transition within the GUI, where the first screen 1 is "folded" in an animated manner, causing the second screen 2 (for example regarding the next feature or content item) to be displayed to the user.

[0202] In certain embodiments, the GUI of the device may include a physical model that takes the user's motion into consideration and generates visual feedback reflecting the user's motion. As an example, once there is an activation input made by the user (for example, in the form of a specific gesture), the user's movement can be continuously tracked through input from one or more of the sensors of the device. Visual feedback can reflect the user's movement in the user interface, while the underlying content remains stationary, so that gestures can be registered and parallax can be used to distinguish between UI features or controls and the underlying content. In certain embodiments, the physical model may include a generalized spring model with damping. In this model, items can be arranged in layers. Deeper layers can have "stiffer" springs in this physical model to keep the item in place. This allows the bottom layer of the user interface to move slightly when the device is moved, while the top layer can move more, creating a sense of parallax. In addition, the spring model can include damping, which makes the motion lagging, creating a more liquid, smooth motion. FIG. 122 illustrates an example of using a physical model in the GUI. The user wears the device 100 on her arm. Once the user moves his arm in a downward manner, the icon 12210 (such as a light bulb) displayed on the screen moves in a manner reflecting the user's movement. However, the underlying content on the screen (such as the background image) does not move. Such floating icons or menu items can, for example, have a display that does not allow the same When displaying the size of many icons or menu items, it is helpful. In addition, this type of floating behavior can also be used with notification means for presenting events to the user.

[0203] In certain embodiments, the GUI of the device may include faces as the default screen or wallpaper of the device, and these faces may be part of the interaction and transformation model hierarchy (for example, in the top layer of the hierarchy or as the main screen). As described herein, these faces can be changeable applications or patterns that can automatically respond contextually to the user's activities. As an example, the face may vary depending on the user's environment, needs, tastes, location, activities, sensor data, gestures, or schedule. The availability of faces (or the transition from one face to another in the GUI) can be determined based on contextual information. As an example, if the user has an upcoming event scheduled in his calendar, the face of the device can be changed to the calendar face that displays information about the upcoming event to the user. As another example, if the user is determined to be near their home (eg based on GPS data), the face of the device may change to the face associated with the home automation application. As another example, if the user is determined to be (for example, based on various biometric sensors, such as heart rate or wakefulness sensors, Or based on the accelerometer) is exercising vigorously, the face of the device can change to fitness mode, showing the user's measured pulse, calories burned, time elapsed since the activity (such as running), and time. Any appropriate sensor data (for example from sensors including biometric sensors, focus sensors, or sensors that can determine the position of the user's hand while driving the vehicle) can be used to determine the context and the appropriate face to display to the user. The user's historical use of the device (for example, a specific time of the day when the user used a fitness application, such as in a fitness class) can also determine which face is displayed on the device. As an example, the device may anticipate the user's demand for the fitness mode at a specific time of the day when the user tends to exercise. Situational facial expressions can also be related to the suppression of notifications (for example, if the user is determined to be driving or if

Device is not worn) or a change in the way the notification is expressed (for example, visually or audibly). In certain embodiments, the face of the device does not need to be associated with any application on the device and may be a wallpaper or background on the display of the device. Faces can be dedicated to specific information channels (such as calendar feeds, health or activity feeds, notifications, weather feeds, or news). As an example, a severe weather notification or reminder (e.g. received from a weather feed) may cause the weather face to be displayed on the display along with the notification. Regardless of the type of face, the face can display the time (for example, in analog or digital format). The face can be user-customizable. The user's customization or taste can be explicitly input by the user (for example, input to the device or management software on the paired device) or directly learned by the device (for example, using sensors and usage data to create a model over time). Figure 123 illustrates example faces, including an analog watch 12310, an analog watch 12320 with a circular menu layout, a health mode face 12330, and a weather face 12340. Figure 124 illustrates a set of example faces 12410-12440 of the device with calendar and appointment information displayed.

[0204] In certain embodiments, the device may be worn on the user's limbs (will not obscure the user's face and do not require the user to hold the device) and may include augmented reality (AR) functions. This AR function can aim the device's camera based on the use of body motion, which can allow for higher accuracy aiming due to the user's proprioception. This type of system may allow the user of the device to view an object in the real world while the user views a version of the object on the display (for example, the version captured by the camera of the device). An example of this AR capability is illustrated in FIG. 16. Such AR systems may allow the use of the "see-through" capabilities of aligned cameras and sensors on opposite sides of the user's limbs. As described in this article, various AR applications can be enabled by this type of arrangement. In certain embodiments, applications can be specifically designed for devices to allow immediate opportunistic use. In addition, a delegation model can be provided on the device to allow the use of external resources to increase the breadth of applications available for running on the device, while incurring fewer (or no) penalties in terms of processing requirements or energy usage. In certain embodiments, the device can control or be controlled by other devices (for example, nearby devices that are discovered via a network and paired with the device in communication). This type of control can be achieved via proximity, gestures, or traditional interfaces. Pairing can be achieved using various technologies including the device's camera, which will be discussed in more detail in this article.

[0205] FIG. 125 illustrates an example of an automatic camera activation decision flow of a device. In certain embodiments, whether the camera is enabled and whether the automatic activation of the camera (for example for object recognition) is enabled may depend on the application or mode the device is currently in. In certain embodiments, automatic camera activation can be enabled on the device 12510_oIf this feature is enabled (determined in step 12520) and if sufficient CPU capacity and power are available on the device (for example to calculate features of interest from the image, which is determined in step 12530), then the camera of the device (for example Outward-facing camera) can automatically capture, process or display 12560 one or more images, if the camera is stably held at the aiming position by the user for a predetermined amount of time (for example, detected by the inertial measurement unit on the wearable device or detected by the image The blur is calculated, which is determined in step 12540). In other embodiments, the camera may always be activated and search for images. In still other embodiments, the camera may only capture an image and perform feature recognition when the user manually triggers image capture (for example, pressing or clicking the outer ring, or tapping the display, which is determined in step 12550). In certain embodiments, when the camera is activated (by any suitable method), the augmented reality (AR) function may be enabled. The AR function can be automatically enabled (depending on, for example, the available CPU capacity and power on the device). In other embodiments, the AR function may be explicitly enabled by the user via any appropriate input made by the user. The user may, for example, provide touch input on the display to enable the AR function. As an example, the user may capture an object such as a bird (for example by pointing the camera of the device at the bird), and the user may touch the image of the bird displayed on the display. This action may enable the AR function of the device, so that, for example, the device recognizes a bird as an object and returns information about the bird to the user. In other embodiments, as described herein, the user can perform one or more gestures to enable AR functions, and use AR functions to perform tasks (for example, by performing typing gestures in the field of view of the device's camera to use a "virtual" keyboard ).

[0206] In certain embodiments, if the device does not have the ability to calculate features of interest on its own, the device can capture an image, transmit the image to a communicatively coupled device (such as a nearby device, such as a phone or a personal computer), or transmit To Internet-based services, where you can remotely calculate the features of interest. Once the features of interest have been determined, one can consult Internet-based services or local data directories to obtain additional information about the identified objects. If the information is found, the relevant data can be displayed to the user along with the identified characteristics on the device.

[0207] Devices may have small form factors in certain embodiments and may be constrained in terms of available memory, processing, and energy. The delegation model may allow the device to delegate some parts of one or more processing tasks (such as tasks related to AR functions) to nearby devices (such as phones or personal computers) or to, for example, network-based or Internet-based services. As an example, for a task that can be delegated, the application that requires the task provides the system (such as the kernel of the operating system of the device) with the characteristics or profile of the task, including the delay sensitivity of the task, processing requirements, and network payload size. This can be done for each delegable subtask of the entire delegable task. Since tasks are often pipelined, successive blocks of the task pipeline can be delegated. In certain embodiments, the system can take measurements of one or more characteristics of the device or build a model of these characteristics. The characteristics of the device may include static properties of the device, such as the properties of the hardware components of the device, including the total installed memory, the maximum CPU speed, the maximum battery power, or the maximum bandwidth of the network interface. The characteristics of the device can also include the dynamic properties of the device, such as the operational properties of the device, including available memory, current CPU capacity, available energy, current network connectivity, availability of network-based services, and average users among one or more users The count of behaviors or the predicted or expected processing time of the task (for example, given a specific usage scenario). In a particular embodiment, let The device may have a model containing previous and current measurements of device characteristics to help determine future device behavior. Based on the task characteristics or profile and these measurements or models, and based on whether the task can be performed on the device, the system can delegate (or not delegate) the task or one or more parts of the task pipeline. For example, if the available memory on the device cannot support the processing of the task (such as playing a video), one or more parts of the task can be delegated. As another example, if the CPU capacity of the device cannot support the processing task (for example, if the CPU is running at maximum capacity due to its existing load), one or more parts of the task can be delegated. As another example, if the battery level of the device is low and the battery is not expected to provide energy to the device for as long as the expected processing time of the task, one or more parts of the task can be delegated. As another example, if the network connectivity of the device is low or does not exist, one or more parts of the task may not be delegated (for example, if the device also has sufficient available memory, CPU capacity, and energy). As another example, if one or more network-based services are available to the device (such as cloud-based services for processing) and the device has appropriate network connectivity (such as good available bandwidth), one of the tasks can be delegated Or multiple parts. As another example, if the user of the device usually (e.g. historically) entrusts the playing of the video, the user can entrust the playing of the video One or more parts of a task. As another example, if the predicted processing time of the task (for example, predicted based on a model containing previous and current measurements of device characteristics) exceeds a certain threshold (for example, a few minutes), the task may be delegated. Any appropriate characteristics (such as static or dynamic properties) of the equipment in any appropriate combination can be used to determine whether to delegate a task. In addition, any appropriate characteristics of the task of the device (for example, including task profile or characteristics of the task, including latency sensitivity, processing requirements, or network payload size) can be used alone or in combination with device characteristics to determine whether to delegate the task . In addition, any model of a device (eg, device behavior) can be used alone or in combination with device or task characteristics to determine whether to delegate a task. In certain embodiments, the device paired with the device may also include a delegation model, so that the paired device (such as a phone) performs the same steps and delegates tasks based on its own model of energy, connectivity, runtime requirements, and feasibility. The delegated task can be processed or run to completion on the paired device (such as a phone), and the result of processing the delegated task can be returned to the device. In certain embodiments, when the device does not have any network connectivity or when the device is set

When there is no paired device in the range of the device, the device can operate in standalone mode (for example, no processing tasks are delegated). Once the device regains connectivity, or when a device is paired with the device, the delegation of the task can continue.

[0208] An example algorithm of the delegation model of the device is illustrated in FIG. 126. In this example, the task process can be delegated to start on the device (12610). The device's system performs power usage analysis and prediction (12620) (for example, based on the user's historical energy usage 12630 and the expected time until the device is charged 12640). Based on this, the system determines in step 12650 whether there is sufficient power remaining for the required running time of the entrustable task. If there is sufficient power remaining, the system of the device can increase the power usage 12660 and handle the entrustable tasks on the device itself 12670_oHowever, if the device does not have sufficient power for the required runtime, the device may query the paired device (eg, phone) 12680 to determine the energy state of the paired device (12690). If in the example of the phone there is sufficient power remaining on the phone for the required run time, the task can be processed on the phone 12694_ΟHowever, if there is insufficient power on the phone, the system may determine in step 12692 whether the device has connectivity to an Internet-based service (such as the cloud) or other network-based service. If not, the device can delegate the process to the phone 12694. If there is connectivity, the device can delegate the process to the cloud 12696, where the tasks are processed and later the results are returned to the device. In certain embodiments, the delegateable task may be delegated by the device to one or more paired devices (for example, mobile phones or personal computers) or network/Internet services in a divided manner. In other words, the delegateable tasks or subtasks of the process can be delegated to different locations by the device. [0209] The present disclosure contemplates that the delegation model of a specific device (or family or range of devices) can be dynamic or contextual. As an example, the delegation model can take into account the available memory, CPU capacity, and available energy of a particular device (or family of devices), all of which can change over time. The delegation model can also take into account the availability of network-based or cloud-based services (and the capacity of each), as well as network connectivity (such as bandwidth and latency), which can also change over time. For example, referring to FIG. 127, according to the first commission model 12710 (which can be applied to devices manufactured next year, for example), most processing can be divided equally between the device and the paired device (such as a smart phone), with only a small number of commissions. Trusted to the server of the cloud-based service. According to the second commission model 12720 (which can be applied to devices manufactured in a three-year period, for example), most of the processing can be handled locally by the device (for example, due to small form factors in the memory, CPU, and energy capacity predicted Progress). In this second model, some processing can be delegated to the server (for example, more than in the first delegation model, due to improved network connectivity) and only a small amount of delegation can occur to locally paired devices. According to the third commission model 12730 (which can be applied to devices manufactured in a five-year period, for example), all or almost all processing tasks can be divided equally between the device and the server of the cloud-based service, and there is no or almost no Processing is delegated to locally paired devices. Any number of delegation models can be created because the factors considered in the delegation model are dynamic. As an example, according to one delegation model, all or almost all tasks can be performed locally on the device, while in another delegation model the device can delegate all or almost all tasks.

[0210] The device can choose to delegate functions to paired processing-rich devices (such as phones, computers, tablet devices, TVs, set-top boxes, refrigerators, washing machines, or dryers) or to the Internet. This choice can be based on energy reserves or go to The connectivity bandwidth of each of these locations is made. For example, a device with a powerful processor can delegate to a paired device when energy is low, or it can choose to delegate to Internet services when the paired device does not have sufficient power reserves. Similarly, if the connection to the Internet exhibits higher latency, the device's system can choose to process it locally to reduce the size of the data transfer.

[0211] In certain embodiments, the entire application or part of the application may be delegated to the paired device by the user of the device, and vice versa. This can happen on a per-application basis. When the application on the target device (such as a TV) is to be entrusted to the device, the target device can send a request through a pairing connection (possibly via an intermediate device, such as a smart phone or a personal computer).

Load the application on the device. The device can then act as a client of a server running on a paired device (such as a TV). Similarly, applications running on the device can be delegated to the paired device (for example, a video played on the device can be delegated to be played on the paired TV). For example, if the device is running a first application, and the user of the device wishes to interact with the second application, the device can automatically delegate the tasks of the first application to be handled by another device (for example, a paired TV).

[0212] FIG. 128 illustrates an example of a decision flow in a device operating according to a delegation model. In this example, the image capture application is running on the device. The scene 12810 is captured on the device, and the device determines whether 12820 has sufficient CPU capacity for image feature calculation. If the device has enough CPU capacity, it will calculate the features of interest in the local computing scene 12830_oIf the device does not have sufficient CPU capacity, it can first determine 12840 whether it is communicatively paired with another device (such as a mobile phone or a personal computer) with greater processing power. If it is paired with such a device, the device can send data to the paired device so that the paired device can calculate the features of interest in the image 12850. If the device is not paired with such a device, it can determine whether it is connected to the Internet-based (Eg cloud) services 12860. If not, the device does not perform further actions. If so, the device can send the data to the cloud service so that the service can calculate the features of interest 12870 in the scene. Any appropriate algorithm, including SURF, for example, can be used to calculate the features of interest (regardless of where it is calculated). their). In this example, the feature of interest can be compared to a local catalog or Internet-based service to determine if any matches are found (and if so, determine relevant information of interest) 12880. If a match 12890 is found, the result can be presented to the user 12895 on the device. If no match is found, no further action is taken.

[0213] In certain embodiments, the camera or other optical sensor of the device can be used to recognize any gestures performed by the user (eg, performed in the space between the camera and a target in the real world). These gestures can be used, for example, to act on presented data (such as real-world targets, such as signs including text) or can be used to point to specific items on which augmented reality functions can be performed. For example, the user can point to a word on the logo, causing the device to translate it and display the translation to the user. Figure 17 illustrates two examples of images captured by the camera of the device. In one example, both the truck 1725 and the user's hand 1720 of the device are within the field of view of the device's camera 1705 and are displayed by the device (shown at 1710). In this way, gestures performed by the user on the truck can be recognized by the device and processed by the device to provide, for example, AR functions. In the second example, only the truck is within the camera's field of view (shown at 1715), so the gesture performed by the user is not captured or recognized by the device. Gesture recognition can also be commissioned by the device.

[0214] In certain embodiments, an object or image can be recognized by the device when it is within the viewing frame of the device's camera. As described in this article, there are many ways for the device to recognize objects. As an example, a gesture performed by a user (for example, a pointing gesture indicating a specific object) can enable the AR function on the device and cause the device to recognize the object. As another example, automatic object recognition may occur when, for example, the user positions the camera on a specific object (such as a piece of text) for a certain amount of time. As a third example, the object recognition or AR function may be explicitly enabled by the user when, for example, the user taps or touches the display (or, for example, clicks on the outer ring) when the camera of the device captures the object of interest. Global object recognition can be computationally intensive and error-prone in some cases. In this way, in certain embodiments, a restricted collection (such as pages of magazines or catalogs or catalogs of certain types of objects, such as plant leaves or book covers) may be applied to improve accuracy. There are several options for calculating feature vectors from images, from which the designer of the device's system can make a choice. In some cases, the conversion of feature vectors between different schemes may be computationally expensive, so that the selection of databases that may match are copied on the device. As described in this article, the calculation of eigenvectors can be commissioned.

[0215] In certain embodiments, various types of barcodes can be recognized by the device. These barcodes can be used to query Internet-based services to obtain additional data, and to purchase, review or bookmark items with barcodes for future use

Review options. Although two-dimensional barcodes can generally be read directly, the system of the device can provide an extra close focus mode for particularly small or one-dimensional barcodes to improve the recognition rate. If the system lacks the ability to decode barcodes, it can simply focus the camera, take a picture, and delegate the recognition to a remote service, as described in this article. Figures 129A-129D illustrate examples of barcode recognition patterns. The device can be pointed to an item (129A), identify the item (129B), display additional information about the item obtained from the Internet (129C), and provide the user with an interface to purchase the item (129D).

[0216] In certain embodiments, the device may perform translation. The translation function can be divided into two parts: optical character recognition (optical character recognition, OCR), and the translation of recognized characters, words or phrases. OCR can be done on the device or delegated (for example, delegated to a paired processing device) to reduce the amount of data that the device needs to translate. Simple word translation can be executed on the device or delegated (for example, delegated to a paired processing device). As with the other functions described in this article, part or all of the recognition or translation process can be commissioned as needed. The user can optionally use gestures to indicate the word to be translated, as shown in FIG. 130 (for example, the word "Warning"). Since individual words can be surrounded by spaces, the system can split words before trying to translate. In addition, if the device can perform OCR with low latency, it can show the text to the user so that the user knows when the device is aiming and correctly recognizes the correct text. If automatic OCR is enabled, the device can automatically recognize the image in the perspective of the outward camera and present display information about the recognized image on the device. If automatic translation is enabled, the device can automatically translate the text in the perspective of the outward camera and present the translated text on the device display.

[0217] FIGS. 131A-131D illustrate examples of devices operating in various augmented reality modes described herein, including barcode recognition mode (131A), image recognition mode (131B), OCR and translation mode (131C), and objects Recognition mode (131D)ο

[0218] FIG. 132 illustrates an example of the overall flow of actions of the augmented reality system of the device. Although this example illustrates an image capture application, any appropriate task or process on the device can follow a similar flow. In addition, any tasks after the device captures the image and before the device displays the results to the user may (where appropriate) be delegated by the device. In this example, an image from the device's camera is captured (in the image capture segment 13210), preprocessed (in the segment 13220), features are extracted and recognized to produce an image recognition result (in the segment 13230), and Any object can be recognized (in fragment 13240). The object data can be formatted for actions taken by the user of the device. The user can activate the augmented reality mode 13211 of the device (for example, via user gestures or pointing the device's camera at an object for a predetermined amount of time), and the image 13212 in the camera's field of view can be captured by the device camera 13213 (for example, based on user input or automatic Trigger events such as camera activation) to generate a camera image 13214_ΟAt this point, the preprocessing stage 13220 can be entered. The pre-processing 13220 may include, for example, contrast enhancement, gray-scale conversion, sharpening, or down-sampling. In certain embodiments, the camera can operate in a general augmented reality mode, in which anything in front of the camera can be processed and recognized. In other embodiments, the camera can operate in a specific mode (for example, OCR, barcode, or visual marking) and only recognize specific items when in this mode. In a specific embodiment, if it is determined that the image may include a known shape, symbol, or organization of the shape or symbol (for example, if the camera or device is in OCR mode, barcode mode, or visual marking mode), then AR image processing may be in the first path Carried on. This first path starts with preliminary processing 13221, proceeds to segmentation 13231 (which can for example determine symbols or symbol group boundaries, such as letters or words), and starts with one or more of the following: optical character recognition 13234 (for example If it is determined that the image can contain characters, then determine what these characters are), barcode recognition 13235 (for example, if the image can Other types of visual markings). The result of this first path is sent to the object recognizer 13242_OIn certain embodiments, if it is determined that the image may include features that are not necessarily known, then AR image processing may be

Proceed on the second path. The second path starts with feature extraction 13222 (for example, where the presence of edges or lines, changes in the angle of lines, edges, points of interest, or patterns are detected in the captured image). The second path proceeds to image recognition 13232, where the characteristics of the image are compared with the characteristic data from the recognition database 13233 (which may exist, for example, on the device, on a locally paired device, or on a remote server or computer) . The result of the image recognition comparison is provided 13237 and sent to the object recognizer 13242. In the object recognition segment 13240, the first and second paths converge at the object recognizer 13242. Here, the result from the object database 13241 is used to identify the object (for example, the phone identified by the image recognition database 13233 is a phone of a specific brand and model). The object data 13243 about the object recognized by the recognizer 13242 (for example, the price of the recognized phone model, or where the phone can be purchased) may be provided. For the text, a definition or translation can appear and be displayed to the user. For barcodes, there may be product information and links to purchase identified objects to be displayed to the user. In certain embodiments, the data may be purely descriptive (for example, the price of the phone) or it may be proactive (for example, a link to where the user can purchase the phone). If the data includes actions Data 13244, the action controller 13250 (which controls, formats, and outputs GUI for the user of the device) can show the user a UI 13255 that includes active data (such as a link for purchasing a phone). If the user selects action 13260 (such as Click on the link), the action controller shows the user the action UI 13265 (for example, the opening of the link), and if the action is confirmed 13270, the action (for example, the actual opening of the webpage associated with the link) is executed 13275.

[0219] FIG. 133 illustrates an example of a network environment. As described herein, in certain embodiments, the device 13310 may be paired with other devices (eg, nearby devices). The device can be directly connected to the personal area network 13320 (which can be bridged to the local area network via other devices on the same network), or the device can be directly connected to the local area network 13330. The personal area network can include, for example, non-WI-FI radio technologies such as Bluetooth, NFC or The ZIGBEEo personal area network may include, for example, a smart media gateway 13322 (for example, a media server), a smart TV 13324, another processing provider 13326, or a phone 13328. The phone 13328 allows the device to connect to the cellular network 13340 and from there to the Internet 13350. The local area network 13330 may include, for example, WI-FL with or without authentication. The local area network may include, for example, a local wireless network router 13332, a smart media device 13334, a smart home appliance 13336, and a home automation technology 13338. The local area network may in turn be connected to the global Internet 13350 via, for example, a local router 13332, which is connected to an Internet service (for example, a proprietary cloud service 13352 or other cloud service partners 13354). Some devices can be reached by the device via direct access (for example, via a personal area network) or via a local area network. The devices reachable by the device can be paired with the device and can be controlled or controlled by the device. system This equipment. The device can be connected to a personal area network or local area network using, for example, any suitable RF technology. As shown in Figure 133, pairing to surrounding target devices may first occur on the RF network. This allows the device to know what is "nearby". This can happen on a personal area network (such as an ad hoc network or a peer-to-peer network) or can use an intermediary network such as 802.11 wireless (such as a local area network). Once the neighbor relationship is established, the device can request nearby devices to enter pairing mode. This can be done directly or via a pairing processing device such as a mobile phone with a wider range of connectivity options. Once the target devices enter the pairing mode, they can display their pairing signals. For example, a device with a display may show visual tags on its display, while others may enable NFC tags to allow the scanner to recognize them. Other schemes can also be used, such as selecting from a list or passing a personal identification code. Once the device is uniquely identified as the pairing target, the device can exchange security tokens with the target device to complete the pairing.

[0220] FIG. 134 illustrates examples of different types of pairing techniques that can be used to pair a target device with a device. The target device, which can be a smart device such as a phone, can include a passive NFC tag 13402 or an active NFC transmitter 13404 (which can be recognized by the devices NFC tag reader 13420 and NFC decoder 13428); NFC decoder 13406 ( It can recognize the NFC tag written by the devices NFC tag writer 13422), passive visual tag

13408 (such as sticker), barcode 13410, or other display information 13412 (which can be recognized by the device's camera 13424); or other pairing system 13416. The active label generator 13414 of the target device can create display information 13412 or provide the information to the target The other pairing system 13416 of the device (which is identified by the mirror pairing system 13426 using the pairing code decoder 13438 of the device). The device can write data to the NFC tag (for example, using the NFC tag writer 13422) to send this data to other target devices that can be paired with the device. The tag written by the device can be recognized by the NFC tag decoder 13406 on the target device. The device can include any of a variety of decoders, including barcode decoder 13430, visual label decoder 13432, image recognizer 13434, or other image-based decoder 13436 (such as flashing patterns for OR codes, logos, or LEDs) Decoder), all of which take input from the devices camera 13424. After the device receives and recognizes the pairing information, it can decode the related information (for example, through various decoders) to start pairing with the target device. In certain embodiments, pairing can use motion to achieve a motion-sensitive target device (such as a mobile phone or remote control) can be paired with the device by holding and moving the target device in the same hand as the device (for example, if two devices are All devices include accelerometers, so similar motion patterns can be detected and used to pair devices). As another example, a fixed target device can be paired with the device by tapping the fixed target device in a random pattern while holding the fixed target device in the same hand as the device (for example, if both devices include touch Detection, similar tapping patterns can be detected and used to pair devices). In addition, pairing can be performed using audio. If both the device and the target device have audio receiving capabilities, the user can make a sound (for example, speak a phrase), and both devices can detect the sound and set up pairing. Any appropriate technology of the device (including, for example, augmented reality functions) can be used to pair with the local device and control the local device. The device and the target device can each be connected to other possible intermediate network devices 13440, and are also connected to the local area network 13450_o

[0221] FIG. 135 illustrates an example process for pairing a target device with a device (eg, using any of the methods described herein). Once the pairing mode is enabled 13510, the device determines whether the RF network contains a pairable target device 13512. If not, no further action is taken (for example, the device can continue to scan periodically). If so, the device may request the pairable device to enter pairing mode 13514. The device may then proceed (in any order, or in a parallel manner) to scan for available target devices via different available technologies. These may include NFC tag scanning 13516, visual tag scanning 13518 in the camera's perspective, barcode scanning 13520 in the camera's perspective, or any other method 13522. If the target device is detected via one of these methods, pair the target device with the device 13524_Ο Once pairing has occurred, the device can show the user menu items for controlling the paired device(s). The device may allow visual gesture control and motion-based gesture control of the paired device. For example, the user may make a gesture (for example, waving her hand) to change the channel on the paired TV, or may make a pinch gesture to transfer video media from the device to the paired display (for example, using the AR function). Device control via the RF network intermediary can be both local and secure. Figure 136 illustrates example controls for paired and controlled TV enablement on the device, including active on/off icon 13610, favorite channel 13620, current channel display 13630, and volume 13640. As described herein, any appropriate input from the user can be used to control the function of the paired device. For example, gesture input, tap or press input, or touch input can be used, for example, to change the channel, adjust the volume, or control other functions of the paired TV.

[0222] In certain embodiments, the pairing and control model for devices may include the following characteristics. The device can act as a host for an application that interacts with a remote device (for example, an application peripheral (appcessory), such as a controllable thermostat) or controls one or more functions of the remote device. The smart phone (or other locally paired device) that may have previously been the host of the application can now only act as a local target device, and the device can delegate certain functions related to the interaction or control of the remote device to the target device (for example, go to Long-distance wireless connectivity of remote devices, sending commands to remote devices

Order, receive data from a remote device or process tasks). The control of the remote application peripheral device can be performed by the device using any appropriate means, for example, including visual means (for example, using a camera) or motion-based gestures. In other embodiments, the locally paired smartphone can continue to act as the host of the application that interacts with the remote application peripheral, but the device can provide some or all of the user interface for data input to the application and data output from the application ( For example, the "light" version of the smart phone hosting application). For example, the user can use the device to control the application, but the smart phone can still act as the host of the application. [0223] In certain embodiments, the device may operate in conjunction with one or more services. These services may fall into categories including security, energy, home automation and control, content sharing, healthcare, sports and entertainment, business, vehicles, and social applications. [0224] Example security applications include the following applications. The device can authenticate the user (the user wearing the unlocked device) to another device near the user (for example, another device paired with the device). The device may be unlocked by a code entered by the user using any suitable input including, for example, the outer ring of the rotating device. As an example, while the user rotates (or presses or clicks) the outer ring, the display may show alphanumeric or symbolic data corresponding to the rotation (or press or click) made by the user. For example, if the user rotates the outer ring in a clockwise direction by one rotation increment (or for example Click or press the outer ring once), the display can show the user "1", and if the user rotates the outer ring in a clockwise direction by two rotation increments (for example, within a certain period of time, such as one millisecond) (or For example, clicking or pressing the outer ring twice), the display can show "2" to the user. In certain embodiments, the display of alphanumeric or symbolic data corresponding to the rotation (or pressing or clicking) made by the user may allow the user to unlock the device using the symbolic method of a combination lock. It is also possible to use biometric data (for example, through the user's skin or bone signature) to unlock the device.

[0225] In an example energy application, the device may automatically display information about the energy consumption of the room or other location where the user is located. The device may also be able to display information about the energy consumption of other paired devices and dynamically update all this information as the user changes location.

[0226] In an example home control application, the user can use, for example, the rotation of the outer ring or gesture input to select and directly control the paired home control device.

[022] The user can use gestures to control the sharing or delivery of content to or from the device (for example, to transmit a video played on the device to a paired TV, as described herein). In addition, the device can provide auxiliary information (such as movie subtitles) for content shown on another larger device (such as a TV screen playing a movie).

[0228] The device can automatically determine the health care situation (for example, whether the user is exercising or sleeping). When it determines this situation, the device can open the application corresponding to the health care situation (for example, for recording heart rate during exercise, exercise during exercise, exercise duration, pulse blood oxygen during exercise, sleep mode, sleep duration Time or galvanic skin response). The device may, for example, measure the user's health-related data (such as heart rate, exercise, or pulse oximetry) and send some or all of this data to the paired device or server. Although it is exemplified in a health care context, the determination of the relevant context (for example, based on the user's behavior), the opening of the corresponding application, the recording of data, or the sending of this data may be applicable in any appropriate context. [0229] The device can assist sports-related applications, such as automatically evaluating a user's golf swing and suggesting corrections.

[0230] In a commercial setting, the device can automatically identify the product when the user picks it up (for example, using RFID, NFC, barcode recognition, or object recognition) and can provide information about the product (for example, nutritional information, source information, or comments) ) Or the option to purchase the product. Payment for the product can be done, for example, using visual barcode technology on the device. In certain embodiments, the device can be used to pay for products using NFC, RFID, or any other suitable form of short-range communication. During payment, the user's information can be authenticated by the device, for example, and the device can detect the user's biometric information (such as bone structure or skin signature). The device may also automatically provide instructions (eg, vibration) to the user when the user is close to a product on his shopping list (eg, stored on the device) or another list (eg, a wish list of the user's friends).

[0231] The device can act as a key for unlocking or opening one or more vehicles. The user can, for example, use the outer ring to enter a code to unlock or turn on the vehicle (for example, using NFC technology), as described earlier. In certain embodiments, unlocking the vehicle may require both user biometric information and a code entered by the user, thereby allowing enhanced security for vehicle-based applications. In addition, the device may include one or more user profiles, each of which contains vehicle settings (such as temperature or seat position). As another example, the biometric information of a specific user can be used not only to unlock the device, but also to determine which user profile to load during operation of the vehicle. The proximity of the device to the vehicle can automatically enable the vehicle to implement the vehicle settings of the user's profile. The device can also be operated for GPS navigation (either directly on the device, or for example when paired with a phone and controlling the phone).

[0232] The device may access a service that supports mixed reality games or massively multi-player reality-based games and operate in conjunction with the service. This function may include, for example, registration, management of user data (such as user profiles and game-related data, such as completed levels or inventory of supplies), and management of achievement lists. The functions of the devices and services may also include the management of connectivity (for example, the hub function), which handles fragile wireless communication channels and provides a unified API to third-party game servers.

[0233] The device can access and operate in conjunction with the following services: the service allows users of the device to post location, sign-in, or other location-based data, which allows various services to access the most current information about the users location and status Consistent warehouse. As an example, users of devices can use similar devices to find friends. Together, the service and the device can handle status updates, profile management, application access permissions, blacklists, or user-to-user access permissions. The service can be a trusted and centralized point of contact for private data. By combining access to the unified location service, energy and battery life can be saved in certain embodiments. In certain embodiments, certain function tokens may be made available based on the user's location. The application can, for example, check on the device to find out if this token is available and act accordingly. On the server side, the API can allow developers to see the use of tokens or allow redemption. In certain embodiments, the information can be distributed by the device to other users (for example, to a single other user, or to multiple users in a broadcast mode).

[0234] The device can access a service that provides a unified voting interface that allows the device to receive and send votes and operate in conjunction with the service. Together, devices and services can manage distribution lists, scoring standards, and voting availability frameworks (for example, both temporally and geographically). This service can be exposed on the device and on the server so that a third party can use the API to write applications and receive the returned results via the online API.

[0235] In certain embodiments, the device may access to provide an optimized service for the presentation of text, images or other information on the circular display of the device and operate in conjunction with the service. As an example, a website can be rendered or formatted for display on a computer monitor, but the service can customize the rendering and formatting for a smaller circular display by emphasizing the image and truncating the text. Customized rendering and formatting can be, for example, tasks that can be delegated between the device and one or more servers or locally paired devices. This service may also include news or advertising services.

[0236] FIG. 137 illustrates an example computer system 13700_oIn a particular embodiment, one or more computer systems 13700 perform one or more steps of one or more methods described or illustrated herein. In certain embodiments, one or more computer systems 13700 provide the functions described or illustrated herein. In certain embodiments, software running on one or more computer systems 13700 performs one or more steps of one or more methods described or illustrated herein or provides functions described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems 13700. Here, where appropriate, reference to computer systems may cover computing devices, and vice versa. In addition, where appropriate, reference to a computer system may encompass one or more computer systems.

[0237] This disclosure contemplates any suitable number of computer systems 13700. This disclosure envisions computers

The system 13700 takes any suitable physical form. As an example and not a limitation, the computer system 13700 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as a computer-on-module -module, COM) or system-on-module (SOM)), desktop computer system, laptop or notebook computer system, interactive digital kiosk, mainframe, computer system grid, mobile phone, personal digital Assistant (personal digital assistant, PDA)> server, tablet computer system, or a combination of two or more of these. Where appropriate, the computer system 13700 may include one or more computer systems 13700; may be unified or distributed; may span multiple locations; span multiple machines; span multiple data centers; or may exist in the cloud, The cloud may include one or more cloud components in one or more networks. When appropriate, one or more computer systems 13700 can execute one or more steps of one or more methods described or illustrated herein without substantial space or time constraints. By way of example and not limitation, one or more computer systems 13700 may execute one or more steps of one or more methods described or illustrated herein in real time or in a batch mode. When appropriate, one or more computer systems 13700 may execute one or more steps of one or more methods described or illustrated herein at different times or at different locations. [0238] In a particular embodiment, the computer system 13700 includes a processor 13702, a memory 13704, a storage device 13706, input/output (input/output, 1/0) interface 13708, communication interface 13710 and bus 13712. Although this disclosure describes and illustrates a specific computer system having a specific number of specific components in a specific arrangement, the present disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement. [0239] In a particular embodiment, the processor 13702 includes hardware for executing instructions, such as those that make up a computer program. By way of example and not limitation, in order to execute instructions, processor 13702 may retrieve (or fetch) instructions from internal registers, internal cache, memory 13704, or storage device 13706; decode and execute them; and then write one or more results to Internal register, internal cache, memory 13704 or storage device 13706_oIn certain embodiments, the processor 13702 may include one or more internal caches for data, instructions, or addresses. The present disclosure contemplates that the processor 13702 includes any suitable number of any suitable internal caches as appropriate. As an example and not limitation, the processor 13702 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLB)_oThe instructions in the instruction cache may be a copy of the instructions in the memory 13704 or the storage device 13706, and the instruction cache may speed up the retrieval of these instructions by the processor 13702. The data in the data cache can be a copy of the data in the memory 13704 or the storage device 13706, for the instructions executed at the processor 13702 to operate on it; the result of the previous instructions executed at the processor 13702 for the processor 13702 The subsequent instructions executed there are accessed or written to the memory 13704 or the storage device 13706; or other appropriate data. The data cache can speed up the read or write operations of the processor 13702. TLB can accelerate virtual address translation for the processor 13702. In certain embodiments, the processor 13702 may include one or more internal registers for data, instructions, or addresses. The present disclosure contemplates that the processor 13702 includes any suitable number of any suitable internal registers as appropriate. Where appropriate, the processor 13702 may include one or more arithmetic logic units (ALU); may be a multi-core processor; or may include one or more processors 13702. Although the present disclosure describes and illustrates specific processing Processor, but this disclosure contemplates any suitable processor.

[0240] In a specific embodiment, the memory 13704 includes a main memory for storing instructions executed by the processor 13702 or data operated by the processor 13702. By way of example and not limitation, the computer system 13700 may load instructions into the memory 13704 from the storage device 13706 or another source (eg, another computer system 13700). The processor 13702 can then load the instructions from the memory 13704 to the internal register or internal cache. In order to execute instructions, the processor 13702

Instructions can be retrieved from internal registers or internal caches and decoded. During or after the execution of the instruction, the processor 13702 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. The processor 13702 may then write one or more of these results to the memory 13704_oIn a particular embodiment, the processor 13702 only executes instructions in one or more internal registers or internal caches or in the memory 13704 (rather than the storage device 13706 or elsewhere) and only executes instructions in one or more internal registers or internal caches. Or data manipulation in the memory 13704 (not the storage device 13706 or elsewhere). One or more memory buses (each of which may include an address bus and a data bus) may couple the processor 13702 to the memory 13704. The bus 13712 may include one or more memory buses, as described below. In certain embodiments, one or more memory management units (MMU) exist between the processor 13702 and the memory 13704 and facilitate access to the memory 13704 requested by the processor 13702. In a particular embodiment, the memory 13704 includes random access memory (RAM). This RAM can be a volatile memory when appropriate, and this RAM can be a dynamic RAM (dynamic RAM, DRAM) or a static RAM (static RAM, SRAM). In addition, this RAM can be single-port or multi-port RAM where appropriate. This disclosure contemplates any suitable RAM. Where appropriate, the memory 13704 may include one or more memories 13704. Although specific memories are described and illustrated in this disclosure, any suitable memory is contemplated by this disclosure.

[0241] In a particular embodiment, the storage device 13706 includes a mass storage device for data or instructions. As an example and not limitation, the storage device 13706 may include a hard disk drive (HDD)>floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive or any of these A combination of two or more. Where appropriate, the storage device 13706 may include removable or non-removable (or fixed) media. When appropriate, the storage device 13706 may be internal or external to the computer system 13700. In a particular embodiment, the storage device 13706 is a non-volatile solid state memory. In a specific embodiment, the storage device 13706 includes a read-only memory (ROM). When appropriate, this ROM can be mask programming ROM, programmable ROM (programmable ROM, PROM), erasable PROM (erasable PROM, EPROM), electrically erasable PROM (electrically erasable PROM, EEPROM), electrically changeable ROM (electrically alterable ROM, EAROM) or flash memory or a combination of two or more of these. The present disclosure contemplates that the mass storage device 13706 takes any suitable physical form. When appropriate, the storage device 13706 may include one or more storage control units that facilitate communication between the processor 13702 and the storage device 13706. Where appropriate, the storage device 13706 may include one or more storage devices 13706. Although this disclosure describes and illustrates specific storage devices, this disclosure contemplates any suitable storage devices.

[0242] In a particular embodiment, the I/O interface 13708 includes hardware, software, or both that provide one or more interfaces for communication between the computer system 13700 and one or more I/O devices. Where appropriate, the computer system 13700 may include one or more of these I/O devices. One or more of these I/O devices may enable communication between the person and the computer system 13700. By way of example and not limitation, I/O devices may include keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet device, touch screen, trackball, video camera, another Appropriate I/O devices or a combination of two or more of these. The 1/0 device can include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces 13708 for them. Where appropriate, the 1/0 interface 13708 may include one or more devices or software drivers, enabling the processor 13702 to drive One or more of these 1/0 devices. Where appropriate, the I/O interface 13708 may include one or more I/O interfaces 13708_οAlthough this disclosure describes and illustrates a specific I/O interface, this disclosure contemplates any suitable I/O interface.

[0243] In a particular embodiment, the communication interface 13710 includes one or more interfaces for communication between the computer system 13700 and one or more other computer systems 13700 or one or more networks (for example, packet-based communication). Hardware, software, or both. As an example and not limitation, the communication interface 13710 may include a network interface controller (NIC) or a network adapter for communicating with Ethernet or other line-based networks, or a wireless NIC (wireless NIC, WNIC) or wireless Adapter, used to communicate with wireless networks such as Wi-Fi networks. This disclosure contemplates any suitable network and any suitable communication interface 13710 for it. As an example and not a limitation, the computer system 13700 can be connected to a self-organizing network, personal area network (PAN)> local area network (local area network). , LAN) >Wide area network (wide area network, WAN)>Metropolitan Area Network (MAN)>Body Area Network (BAN) or one or more parts of the Internet or a combination of two or more of these for communication. One or more parts of one or more of these networks may be wired or wireless. As an example, the computer system 13700 can be connected to a wireless PAN (wireless PAN, WPAN) (such as Bluetooth WPAN), WI-FI network, WI-MAX network, and cellular telephone network (such as Global System for Mobile Communications, GSM) Network) or other appropriate wireless network or a combination of two or more of these. Where appropriate, the computer system 13700 may include any suitable communication interface 13710 for any of these networks_oWhere appropriate, the communication interface 13710 may include one or more communication interfaces 13710. Although specific communication interfaces are described and illustrated in this disclosure, any suitable communication interfaces are contemplated by this disclosure.

[0244] In a particular embodiment, the bus 13712 includes hardware, software, or both that couple the components of the computer system 13700 to each other. As an example and not a limitation, the bus 13712 may include an accelerated graphics port (Accelerated Graphics Port, AGP) or other graphics buses, an enhanced industry standard architecture (Enhanced Industry Standard Architecture, EISA) bus, and a front-side bus (FSB) , Hypertransport (HT) interconnection, Industry Standard Architecture (ISA) bus, INFINIBAND interconnection, low-pin-count (LPC) bus, memory bus, microchannel architecture (Micro Channel Architecture, MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI Express (PCI-Express, PCIe) bus, serial advanced technology attachment, SATA) bus, Video Electronics Standards Association local (VLB) bus, or other appropriate bus or a combination of two or more of these. Where appropriate, the bus 13712 may include one or more buses 13712_OAlthough this disclosure describes and illustrates a specific bus, this disclosure contemplates any suitable bus or interconnection.

[0245] Here, where appropriate, the one or more computer-readable non-transitory storage media may include one or more semiconductor-based or other integrated circuits (ICs) (such as field-programmable gate arrays (field-programmable gate arrays, ICs)). programmable gate array, FPGA) or dedicated IC (application-specific IC, ASIC))> hard disk drive (HDD), hybrid hard drive (HHD), optical disc, optical disc drive (ODD) )> Magneto-optical disk, magneto-optical drive, floppy disk, floppy disk drive (FDD), magnetic tape, solid-state drive (SSD), RAM drive, secure digital card or drive, any other suitable computer readable Non-transitory storage media, or any suitable combination of two or more of these. Where appropriate, the computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0246] In this article, "or" is inclusive rather than exclusive, unless expressly indicated otherwise or contextually

Otherwise instructions. Therefore, in this article, "A or B" refers to "A, B, or both", unless expressly indicated otherwise or otherwise indicated by context. In addition, "and" is both joint and individual, unless expressly indicated otherwise or otherwise indicated by context. Therefore, in this article, "A and B" refer to "A and B, jointly or individually", unless expressly indicated otherwise or otherwise indicated by the context.

[0247] The scope of the present disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that those of ordinary skill in the art will understand. The scope of the present disclosure is not limited to the example embodiments described or illustrated herein. In addition, although the present disclosure describes and illustrates the various embodiments herein as including specific components, elements, features, functions, operations, or steps, any of these embodiments may include those of ordinary skill in the art. Any combination or substitution of any components, elements, features, functions, operations or steps described or illustrated anywhere in this document. In addition, it is mentioned in the appended claims that a device or system or a component of a device or system is adapted, arranged, capable of, configured, enabled, operable, or operated to perform a specific function Covers the device, system, or component, regardless of whether it or the specific function is activated, turned on, or unlocked, as long as the device, system, or component is so adapted, arranged, capable of doing so, configured, or used in this way Can, can do this, or do this.

[0248] Although the present disclosure describes specific structures, features, interactions, and functions in the context of wearable devices, the present disclosure contemplates that these structures, features, interactions, or functions can be applied, used, or used as appropriate. Used in any other appropriate electronic devices (such as smart phones, tablet devices, cameras, or personal computer devices).