Attention-based interaction in a virtual environment

Attention-oriented interaction in a virtual environment

Related applications

The US patent number of Doreen Y. Cheng (PHA 23-349), entitled "Priority-based Virtual Environment" and filed together, is incorporated herein for reference.

Background of the invention

The present invention is related to a set of systems, devices, and methods that enable participants to interact in a virtual environment, in particular to a set of systems, devices, and methods that enable participants to participate in interactions that focus on attracting attention in a virtual environment.

A large number of consumer groups in the electronics market are dealing with the integration of computer-based development and traditional audio-visual products. In the computer-based development, network systems (ie, the Internet) provide new opportunities in different fields, including entertainment, education, information search, and social interaction. Traditional audio-visual products provide convenience in use and comfort and welcome of consumers in the family, especially in family entertainment rooms.

An integrated approach is to rely on virtual environments. The virtual environment includes the background or scene of the material produced by the computer, which is generally a simulated or real three-dimensional representation of the space. Such a background/scenery can be a city, a boulevard, a single store, a building, an office suite, a single office or some other space. This kind of presentation can be very lifelike or somewhat lifelike in terms of the description of the real world and the human sensing it supports.

In any case, a virtual environment is usually composed of virtual objects, which traditionally include living or living entities. Inanimate entities can include environmental features, such as the walls of a virtual office, where there will always be inanimate walls. Living entities can include so-called avatars and "bots". Broadly speaking, "Bart" is generally an image generated by the participant himself, which performs a preset task or provides characteristics in the environment. A "bart" can include, for example, a wall that transfers and imports information. In contrast, an avatar is generally an image that represents the participant and is controlled by the participant. Traditionally, an avatar provides one or more body gestures, facial expressions, speech, and actions.

However, a virtual environment is generally equipped to induce participants to fall into the sensory experience they provide. To achieve this, participants interact with environmental objects. To give an example, the social interaction between participants is carried out through the interaction between the avatars of the participants; for example, when walking in a virtual park, when the physical path meets, the interaction will unfold, and another For example, a participant can direct his avatar to interact with a "Bart". In this way, the participant can obtain certain required information through this interaction. In both cases, participants generally interact through their avatars.

Generally speaking, the quality of participant experience depends on the quality of interaction supported by the environment. In this relationship, existing research on virtual environments has pointed out that the quality of social interaction between participants is a dominant factor that determines whether participants are motivated to visit a virtual environment repeatedly. These studies have also shown that the quality of social interaction depends on the support of facial expressions and other body language in the avatar that are generally related to face-to-face encounters in the real world.

Although the results of the study are so; however, the current virtual environment cannot support or insufficiently support the social interaction between participants. In order to improve the quality of social interaction provided, various challenges will be met. One of the challenges is to give a natural and socially acceptable agreement to the interactive dialogue. This type of agreement will enable participants to sense and/or become very socially comfortable in the virtual environment. However, in general, traditional virtual environments cannot support such agreements. To give an example, traditional virtual environments tend to overemphasize the ability of one to participate in order to get the attention of other participants. Such agreements are neither natural nor fully accepted by the society. As another example, traditional virtual environments tend to improperly support paired conversations in group social settings (for example, a group of people). In these settings, if the conversation is via text (ie chatting), then when one or more participants speak, the message is usually displayed on a fixed part of the display. If the conversation is conducted by voice, then multiple participants can talk at the same time, and the volume is roughly the same. According to the above two cases, the messages of each person speaking have a tendency to be confused. What's more, the specific message of a specific object held by a participant may be missing, that is, the message disappears before the participant recognizes the message.

In addition to granting appropriate agreements, another challenge related to supporting social interaction in a virtual environment is the management of resource loading. Especially in the communication and provision of multimedia data, traditional virtual environments tend to be over-loaded and/or improperly allocated resources. Such over-loading and improper configuration usually hinder the support of social interaction through multimedia in order, such as through conversation, music, environmental sound, facial and posture vitality, and/or video.

Therefore, for virtual environments, a system that can overcome the shortcomings of traditional virtual environment technologies is needed, especially for social interaction, individual and collective expression, and data communication.

Summary of the invention

The object of the present invention is to provide a virtual environment that overcomes the shortcomings related to social interaction.

Another object of the present invention is to provide a system that completes the social interaction between objects in the virtual environment that is focused on attracting attention, and at the same time supports priority-based expression and data communication in the virtual environment.

According to an aspect of the present invention, a system is provided that enables participants in a virtual environment to easily select a partner, and interact with the selected partner via individual objects (ie, avatars). The system includes an avatar attention element. The avatar attention element is mainly to attract attention, and an attention sensing space and an attention focus space are used in between. The sensing space is composed of a sensing cone contained in the sensing ball, and a sensing vector is located along the center line of the sensing cone. The attention focus space is composed of a focus cone contained in the focus sphere, and there is a focus vector along the center line of the focus cone. For example, the sensing cone is a visual cone, which defines an optical centerline, along which a virtual environment appears in front of the participant's eyes. The focal cone may be coaxial with the visual cone, and it is a sub-cone of it. Participants control the aforementioned two cones by moving, for example, a head-mounted display or a mouse.

In a specific embodiment, the attention element is combined with the avatar vision. The attention sensing space is a scanning space composed of a visual cone contained in a visual sphere, and a visual vector along the center line of the visual cone. The focus space of attention is a gaze space, which is composed of a gaze cone contained in the focus sphere, and a gaze vector along the center line of the gaze cone.

In another specific embodiment, the attention element is combined with non-visual sensing (e.g., hearing). In another specific embodiment, multiple attention sensing spaces are provided, for example, one of them is combined with vision, and the other is combined with non-visual (for example, hearing). In the latter case, each type of sensing is best combined with the individual sensing and focal space.

The avatar notices that the components are optimally assembled to support planned and/or spontaneous interactions; planned interactions, whose interactions are driven by defined clear controls; spontaneous interactions, whose interactions are caused by non- Driven by planned accidental encounters, such as spontaneous "eye" contact between avatars. When two objects are in each other's respective focal spaces, spontaneous interaction can occur in other situations. The avatar attention element makes the best use of the sensing and focus space, especially its cone and vector, so that the combination of calculation and attention can be given fairly quickly. The avatar attention component best uses an interactive control engine to give dynamic and enhanced control of objects in the environment, such as the position change and attention drift of an interactive component, and the interference of non-interactive objects.

The system also optimally includes a priority element. The priority component determines the priority of avatars and objects. The highest priority avatar/object has (a) enhanced processing of the sensing and range supported by the system (for example: enhanced resolution, frame rate, color, and/or other) Quality), and (b) improved network and operating system service quality. The priority element responds to one or more of the following selected parameters, including, for example, (i) the relative position of the object, (ii) the direction, orientation and amplitude of the sensing and focal space, (111) the participants profile, (iv) ) Parameters preset by developers or participants, (v) social parameters, (vi) economic models, and (vii) combinations of these and other selected translation parameters. The result of priority designation not only reduces the overall complexity (such as reducing system loading related to multimedia computing and communication), but can also be combined with social factors, including factors related to interaction.

In some specific embodiments, this system only uses attention components. In other specific embodiments, the system uses both attention components and priority components.

The various novel features of the present invention are described in detail in the scope of the patent application, which is appended and constitutes a part of this specification. In order to better understand the present invention, its operating advantages, and the specific objects achieved by its use, please refer to the drawings and descriptions, which illustrate the best specific embodiments thereof, with similar reference numbers used to identify the same or similar elements.

Detailed description of the invention

<u>the term</u>

The virtual environment system used here refers to an independent or multiple, centralized or decentralized computing-based system that can support a virtual world or other virtual environments, or install/sensing from it, especially any Such an environment illustrated by avatars, "Bart" and other virtual objects.

A virtual object (or sometimes just written as an object) is any thing or part of the virtual environment, whether it is animate, inanimate, or a combination of the two. Inanimate virtual objects may include environmental characteristics, such as an inanimate wall in a virtual office. Living virtual objects may include avatars and "Bart".

The avatar used here refers to a virtual object that represents the participant and is controlled by the participant. An avatar traditionally supports, for example, one or more physical gestures, facial expressions, conversations, and movements. Although an avatar is so representative and supportive, in addition to being personified, it can also include a combination of animate and inanimate virtual objects.

The non-incarnation object used here means a virtual object that does not represent a participant. Traditionally, non-avatar objects can be operated automatically from the participant, support preset sensing (for example, similar to the avatar's vision), have preset functions (for example, provide information), and perform preset tasks (for example, , Collect information) or provide characteristics of the environment (for example, an image of a tree). A non-incarnation object is also considered to be a component of an incarnation. For example, a non-incarnation object can include a wall, and when it interacts with an avatar, it is still a wall or is alive and can directly send a message to the interactive avatar. For another example, an incarnation object It can include an appendage or other parts of a first avatar. When the second avatar triggers the first avatar, the first avatar will provide information selected by the second avatar, such as the participants information combined with the first avatar. Personnel information.

The planned interaction used here refers to those interactions in the virtual environment governed by presets. Explicit control facilitates the interaction, and this type of control is actively selected by a participant. These interactions are similar to the real-world "tapping on the shoulder" to attract attention. Then in this virtual environment, in a general concept, the planned interaction is an object that will definitely attract the attention of other objects.

The spontaneous interaction used here refers to those interactions in the virtual environment governed by a sense or a combination of multiple senses of an encounter between related objects, which is triggered by a preset and automatic mechanism. Interactions generally do not use explicit controls.

The interactive dialogue used here refers to any state in which an object interacts with another object in the virtual environment. When an object is starting, established, executing, and/or ending its interaction with other objects, it is engaged in an interactive dialogue (here sometimes refers to being "occupied" by another object). When an object senses another object but does not participate, then it is not engaged in an interactive dialogue (see Figure 8 for the basic state).

<u>Summary</u>

As shown in Figure 1a, as considered in the present invention, a virtual environment system 8 includes a network system 10 through which the system is combined with one or more hosts 20 through a communication channel 12, and each host 20 The part is connected to a participant 14 who interacts with the virtual environment through his own station 20.

The network system 10 includes a selected topology. An example of the topology of the network system 10 includes a star network having a centralized computing system and connecting all hosts 20. The topology in the example provides: (i) the centralized computing system running the server software, which manipulates the virtual environment (such as importing or exporting the interactions, actions, and other characteristics and behaviors of the selected virtual objects). Data); (ii) The host 20 executes client software that controls the local virtual experience, including interacting its participants with or in the environment ("interactive data") and multimedia data (such as images, Importing or exporting related data such as sound and/or text to the server; and (iii) the server software controls the distribution of data among all hosts 20, including interactive data received from each host 20. It is understood that the centralized computing system can be composed of a host 20.

Another example of the topology used for direct connection between hosts 20, where (i) a typical connection that relies on protocols such as transmission control protocol and/or user data message protocol, (ii) server/host software is basically Distributed among each host 20, and (iii) the software of each host sends the interactive data of its participants to other hosts 20 relative to each other. To give another example of the topology used in direct connection, a participant spreads its data to a group of selected addresses (ie, multiple transmissions) instead of spreading interactive data one after another among all participants. This type of specific group Each host 20 of the address can choose whether to accept/reject the data. It can be confirmed that other topologies can be used without departing from the principle of the present invention, for example, a topology including one or more characteristics of the above-mentioned topologies.

According to the present invention, a host 20 includes a computing device 22 and is connected to each device such as one or more display devices 24 and one or more input/output devices 26 through a connection 21. The wiring 21 typically includes wires/cables, but as for the connection of one or more of the input/output devices 26 and the display device 24, wireless technology (infrared technology) can be used.

In addition to observing the virtual environment 16 through the display device 24 (refer to FIG. 1 b ), the participant 14 interacts with the virtual environment 16 via an interactive connection 36. The interactive wiring 36 provides a man-machine type link between the participant and the input/output device 26 and/or the display device 24.

Each display device 24 has a screen 28, which maps the environment 16, especially the related scenery/environment. As in a specific embodiment shown, the screen 28 maps the participant's avatar 18, other avatars 17, and an inanimate object 19 (for example, a flash of light that strikes the avatar 18).

As depicted, although the system 8 provides a third-person view of the environment, it can be confirmed that the system 8 can provide a third and/or first-person view without departing from the principles of the present invention. In other words, the system is optimally completed, so that the virtual environment is presented to its participants in the first person or the third person's field of view: (i) The first person's field of view provides feelings and is effectively used in the virtual environment Interact as if the participant is the avatar (for example, as if the participant is effectively viewing the environment through the eyes of the avatar) and (ii) the third persons field of vision provides feeling, and separates from the avatar and interacts in the virtual environment (For example, it seems that the avatar can be viewed effectively through a camera set up in a field of view, so that the avatar can be observed in the environment). However, it is better that the participant can feel and interact with any point in the virtual environment in time through only one field of view (the "control field of view").

A preferred display device 24 includes a head-mounted display equipped with spatial configuration technology (ie, detects the movement of the head and/or eyes relative to, for example, the displayed virtual environment 16). However, it is understood that other types of display devices 24 (for example, a single or an array monitor) can be used without departing from the principles of the present invention.

The input/output device 26 preferably includes one or more data acquisition devices, which provide the interaction of the participants and the data in the virtual environment. In this way, the input/output device 26 can better respond to the participant's limbs or social actions based on sensing. It can be confirmed that the device can include a single item or a combination of multiple items, a keyboard, a microphone, an audio speaker, an olfactory generator/detector, and a pointing device (such as a mouse, track Balls, touch pads, and/or joysticks), motion sensors, facial gesture detection devices, virtual reality gloves, specific clothing or other clothing, and other data acquisition devices.

The computing device 22 typically includes a general-purpose computer 29 equipped with a processor 30, a memory system 32 (for example, volatile and/or non-volatile memory), and supporting software 34. The software 34 preferably includes a set of operating system 33, a set of virtual environment components 35 (for example, a host, a host/host or other software combined with the topology of the environment), and some other application programs 37. The operating system 33 preferably supports multimedia, for example, through a graphical user interface. The operating system 33 typically includes a set of component programs. The operating system component program generally includes (i) a device driver combined with the individual input/output device 26, and (ii) a display device driver combined with the display device 24.

It is understood that the present invention allows a wide range of operating system structures. Therefore, no special commercial operating system or structural feature is more suitable for implementing the present invention. Furthermore, it can be confirmed that, in addition to the operating system described above, any completed operating system that does not deviate from the principle of the present invention can be used. For example, the operating system 33 can be omitted, different device drivers can be combined or reinstalled, and new components can be added or not added.

Although the system 8 is depicted as having multiple hosts 20, each of which includes a computing device, and each of them is integrated with the network system 10 rather than separated, it can be confirmed that the others do not deviate from the original The method of accomplishing the principle of the invention can also be used. For example, in a specific embodiment, the system 8 only includes a host 20 for one or more participants to use through one or more display devices 24, where the host 20 provides the network system 10.

As depicted in FIG. 1b, a virtual environment system 8 includes an avatar attention element 52 and a priority element 54. In a specific embodiment: (a) The avatar attention component 52 includes an attention module 38 and an interactive control engine 39, the virtual environment component 35 and related hardware (whether configured in one or more hosts 20 And (or) in the network system 10) and software (e.g., parts involved in the operating system 33, other application software 37, and one or more hosts 20 and/or any other components of the network system 10) (Software), (b) the priority component 54 includes a priority module 40 of the virtual environment component 35, and related hardware (whether configured in one or more hosts 20 and/or in the network system 10) ) And software (for example, the part involved in the operating system 33, other application software 37, and one or more hosts 20 and/or any other software of the network system 10). However, in other specific embodiments, whether it is one or both of the attention component 52 and the priority component 54, it may not have one or more of the modules 38, 39, 40, and/or other relative components. It can still be done.

The avatar attention element 52 is mainly to attract attention, wherein it uses one or more attention spaces 55, and each attention space 55 includes an attention sensing space 56 and an attention focus space 58. The attention sensing space 56 preferably includes a sensing measurement (such as a sensing ball 62) and a sensing measurement portion (such as a sensing cone 60). As depicted, the sensing cone 60 is included in the sensing ball 62 and includes a sensing vector 64. The attention focus space 58 preferably includes a focus volume (for example, a focus ball 68) and a focus volume portion (for example, a focus cone 66). As depicted, the focus cone 66 is included in the focus ball 68 and includes a focus vector 70. Preferably, the focus cone 66 is also included in the sensing cone 60. (Sensing and focus balls are sometimes individually or collectively denoted as "attention balls" here; sensing and focus cones are sometimes individually or collectively denoted as "attention cones" here; and sensing And the focus vector here is sometimes individually or collectively expressed as the "attention vector".)

The attention element 52 incorporates one or more senses. (Examples of typical sensing include sight and hearing.) As for a special sensing, the attention element has an attention space. The sensing space 56 of the space is preferably used to control the object that the participant/avatar is activated (for example, via processing) in the environment to sense. The next one is that the focal space 58 of the space is preferably used to determine the prospective partner (such as an avatar and/or other virtual objects) and which participant's avatar is activated (such as via focus) to interact spontaneously. In this interaction, the prospective partners are preferably avatars and other objects that can be sensed through the sensing space 56 and located in the focal space 58. In Figure 1b, all the avatars 17a-c are located in the attention space 55 of the avatar 18, but (i) only the avatars 17a and 17b are located in the attention sensing space 56 of the avatar 18, and (ii) only the avatar 17a Located in the focus space 58 of the avatar 18. (As explained below, these two spaces also preferably provide priority-based processing of avatars and other virtual objects.)

Therefore, the avatar attention element 52 considers the use of multiple spaces. The first space is the virtual environment. In this space, none, part or all of the existing avatars and other virtual objects in the environment are detectable for each of the supported sensing. Other spaces include individual attention spaces, each of which is preferably combined with one of the supported sensing. The attention space is preferably selected for individual selection (for example, by sensing) relative to the size and configuration of the virtual environment space. Furthermore, the attention space preferably includes the sensing and focus spaces as described, and these selected spaces have better dynamic orientations in the attention space, and have the selected spatial degree (for example, two degrees). Space, three-dimensional space, etc.).

<u>Pay attention to components</u>

The following discussion illustrates the visual aspects of the avatar's attention component. It can be confirmed that other sensing is generally provided by the attention component, and such support is similar to the following visual details.

(In the rest of this document, vectors are represented by underlined marks.)

As depicted in FIGS. 2 and 3, in the context of vision, the attention space 55 of the attention element has (i) a sensing space 56 including a scanning space 100, and (ii) a gaze space 150offocus space 58. Referring to FIG. 2, the scanning space 100 includes a vision cone 102 having a vision vector 106.

The vertex "A" 108 of the cone 102 is combined with the selected immersion point 103. With a third person's field of view, the immersion point 103 is typically separated from the avatar 101, such as a viewing point. As shown, viewing an avatar 101 from the first person's field of vision, the immersion point 103 is preferably a part of the avatar 101. For example, in the visual field of the first person, the immersion point 103 is preferably arranged in the center of the face 110 of the avatar. However, the immersion point 103 may be arranged in the first three positions (for example, "eyes") of the face 110 of the avatar or combined with the limbs 112 of the avatar. What can be confirmed is that these examples rely on the avatar having a face, eyes, and/or a limb. In fact, in any particular completion, the avatar may lack one or more of these characteristics.

It can also be confirmed that in any particular completion, the attention vector is preferably combined with the selected immersion point 103. However, the attention vector preferably points to the immersion point 103. In the first person's field of vision, because the immersion point 103 is combined with the part of the avatar 101, the attention vector is also combined with the part of the avatar, and fully points to the surface of the part.

The visual vector (<u>AA"</u>) 106 is collinear with the meridian 114 of the cone 102, and has the result of |<u>AA"</u>The length indicated by the value of |. The vector 106 belongs to the three-dimensional space of a visual ball 104. In the field of vision of the first person, the avatar 101 is preferably arranged in the center of the visual ball 104. From the perspective of a third person, the participant is preferably actually configured as such. As described below, the direction of this vector indicates the center of the actual attention, while its length limits the distance to the range where the attention can be directed.

Referring to FIG. 3, the gaze space 150 includes a gaze cone 152 having a gaze vector 154. In any field of view (first or third person), the gaze cone 152 and the related vision cone 102 share the vertex "A" 108 and the immersion point 103.

The gaze vector (<u>AA'</u>) 154 includes the warp 114 of the cone 102, and is provided by |<u>AA'</u>The length indicated by the value of |. The vector 154 is preferably collinear with the visual vector 106 and the direction is along the axis 114. However, it is understandable that other directions of the vector 154 can be selected without departing from the principle of the present invention. The length of the gaze vector 154 is typically less than or equal to the length of the visual vector 106 (i.e. |<u>AA'</u>|＜=|<u>AA"</u>|) And the inner angle of the staring cone 152 is preferably less than or equal to the inner angle of the visual cone 102. As described below, the direction of the gaze vector indicates the center of the focal point, while its length limits the distance to the range where attention can be focused.

The scanning and gaze spaces 100 and 150 are preferably bounded. The gaze space 150 is preferably surrounded by two surfaces: (i) the surface 156 of the gaze cone 152, and (ii) the surface 160 of the gaze ball 158. Similarly, the scanning space 100 is preferably surrounded by two surfaces: (i) the surface 116 of the visual cone 102 and (ii) the surface 118 of the visual ball 104. The center of the gaze and vision spheres 158 and 104, the immersion point 103, is combined with the apex 108 (ie, "A") of the respective cones 152 and 102. The radius of the gaze ball 158 is the length of the gaze vector 154 (i.e. mid|<u>AA'</u>|); The radius of the visual ball 104 is the length of the visual vector 106 (ie |<u>AA'</u>|). With the first person's field of vision, the boundary of the gaze space 150 is depicted in FIG. 3, and the boundary of the scanning space 100 is depicted in FIG. 2. A similar boundary applies to the space with a third person's field of vision.

Because the length of the gaze vector 154 is typically less than or equal to the length of the visual vector 106, the gaze space 150 is typically contained by the scanning space 100, as depicted in FIG. 4. However, because the preferred portions of the spaces 100 and 150 are surrounded by the respective balls 104 and 158, the spaces 100 and 150 preferably include the conical portions of the respective balls 104 and 158. In addition, these balls preferably include a boundary volume, in which the associated attention cone can therefore be extremely enlarged in the individual sphere (ie by increasing the internal angle of rotation of a cone), and not be used in this volume. Any changes in order to fully surround all or complete individual balls.

Although the system 8 can sense visual signals combined with virtual objects located in the visual sphere 104, the boundaries of the scanning space 100 typically establish actual limits for each sensing (e.g., typically processing is to limit a predetermined Space). Then, the gaze space 150 establishes a practical limit on the focus of the gaze. Regarding the visual field of vision of the first person, for example, the surface 118 of the visual ball typically defines the maximum sensing limit based on the avatar, while the surface 116 of the visual cone defines the actual sensing limit based on the avatar. Then, the boundary of the gaze cone effectively limits the avatar-based virtual focus with respect to other avatars and other virtual objects. In one completion, to support the first person's field of vision according to this example, the participant preferably initiates a spontaneous conversation with the avatar and other objects in the gaze space 150 of the participant's avatar.

The use and attention to the boundaries of space, sphere, cone, vector and similar objects can be better selected and translated to suit the first person's field of vision, the third person's field of vision, or both. If the vision of the two is completed, it is better that they are independent configurations. For example, the first-person field of vision can be completed to combine every attention space, sphere, cone, vector, vertex, and immersion point, while the third-person field of vision can be combined with some or all of the above objects.

If the two fields of view have been completed, it is preferable that when one field of view is selected, the attention space controls attention (the "control field of view"). In this situation, the attention space of other visual fields is cancelled or invalidated (that is, dependent and consistent operations). However, it is understandable that the field of view can be completed so that when one field of view is selected for translation, the attention space of the other field of view is not cancelled or invalid (that is, the space has independent and consistent operations regarding attention).

Thus, in terms of vision, a scanning space typically illustrates the limitations of the participant's sensing of the virtual environment. With the first-person field of view, the participant sees through the "eyes" of their avatar, and can only perceive the virtual objects that exist in the scanning space 100 through vision. With a third person's field of view, the participant is preferably not bound by the scanning space 100 of the first person's field of vision; the immersion point is typically separated from the participant's avatar, and the suitable scanning space is preferably An optional part of the virtual environment. Furthermore, the elements of the third person space (such as balls, cones, etc.) and the relative elements of the first person space preferably have a selected dimension and spatial relationship. It is understandable that in other relationships, (1) the third persons visual ball can be larger, smaller or equal in size than the first persons visual ball, and (ii) a supporting third persons visual cone can be part of Participation includes, all includes, or does not include the first person's visual cone, and (iii) the third person's visual field support attention cone may partially include, all or not include the participant's avatar. In a specific situation, the supported third-person space is completed so that the participant can not only sense what they will sense in the first persons field of vision, but also sense the supplementary virtual environment here. The selected part.

One advantage of supporting the third person's field of vision is to further enable the participant to pursue interaction with virtual objects, especially those outside the first person scanning space 100 configured in their avatar. As an example, the participant can use explicit controls to initiate planned interactions with objects outside the first person scanning space 100. In addition, the participant can manipulate their avatar to approach the selected object in order to establish a spontaneous interaction related to the situation. However, if the third-person field of view is completed to include a gaze cone, then spontaneous interaction can be supported in the third-person field of view.

Although the cone 60 of the sensing space is suitable for the attention element, in terms of vision, it can be confirmed that the cone 60 may or may not be suitable for other aspects of sensing, and is subject to application and/or content development. Give an example to compare the visual example in this document. The application can complete the hearing. This is a general special case: a participant can sense sounds from a wide range of configurations (for example, from the immersion point All directions and any distance within the maximum range, where the immersion point is located on an avatar in the first persons field of vision, or separated from an avatar in the third persons field of vision), regardless of the direction of the participants ears, and the absence (or Completely lacks) the sensitivity of direction. In this way, the participant can generally point out the source of the sound quite accurately. Therefore, an attention space 55 combined with hearing is preferably completed in one of the following situations: (i) there is no sensing cone 60, or (ii) there is a sensing cone 60 that is connected to the sensing ball 62 Co-expansion or greatly co-expansion (that is, the inner angle is at or close to 180 degrees).

<u>Staring at the situation</u>

In a virtual environment, the avatar attention element 52 preferably provides: (i) when a participant A gazes at an object O (called unilateral gaze), A can interact with O, and (ii) when a participant When person A stares at avatar B, and participant B stares at avatar A (called unilateral gaze), A can have an interactive conversation with B. Regarding gaze, the avatar attention component provides that the participant A is gazing at an object O or avatar B, and the display object O/avatar B is arranged in a gaze cone of A. Similarly, the avatar attention component provides that participant B stares at an avatar A, which shows that the avatar A is arranged in a gaze cone of B. (In this document, italicized capital letters such as A sometimes indicate a participant or an avatar of that participant, or both.)

Preferably, the virtual environment supports gaze in the context of the first person and (or) the third person's field of vision. For example, the avatar attention component can be completed to provide various conditions under A and B gazing at each other, including: (i) avatar B is placed in the first person gaze cone of A, while avatar A is placed in the first person of B In the gaze cone; or (ii) Avatar B(A) is placed in the first person gaze cone of A(B), while the avatar A(B) is placed in the first person gaze cone of B(A); or (iii) ) The avatar B is arranged in the third person's gaze cone of A, and the avatar A is arranged in the third person's gaze cone of B. Especially in most of the later gaze situations, it is preferable that the third person's gaze cone is cut to provide sufficient separation between objects. If the control field of view is completed, then the above-mentioned mutual gaze situation is as normal: Avatar B is placed in A's control field of view gaze cone, while avatar A is placed in B's control field of view gaze cone.

For the sake of brevity, the description of this patent application sometimes includes the gaze in the first person's field of vision. However, it is understandable that the subject of extending the field of vision to a third person and extending to the field of control can be easily appreciated by ordinary people.

FIG. 5 shows an example of a first avatar 200 gazing at a second avatar 202 in a virtual environment 201, that is, the second avatar 202 is in the gaze cone 204 of the first avatar. However, when the first avatar 200 is located outside the gaze cone 206 of the second avatar, the second avatar 202 does not stare at the first avatar 200. FIG. 6 shows an example in a virtual environment 201, in which the first and second avatars 200 and 202 are staring at each other in the first person's field of vision, and an interactive dialogue is thus initiated. In this situation, because each avatar is arranged in the focal point of the other party, the first avatar 200 and the second avatar 202 can stare at each other.

The configuration of avatars, objects, immersion points, and similar things can be determined in various ways. As an example, the configuration of an avatar can usually be determined by the configuration of the item points of the first person focus cone of an avatar, because each such vertex is typically combined with an immersion point (for example, face or eyes) of the avatar . In this way, when the apex of a selected focus cone of an avatar is located in the first or third person gaze cone of a participant, the avatar is located in the first or third person gaze cone of the participant. However, as another example, the configuration of an avatar, and a particular object without an avatar, can be determined independently of any cones and/or vertices. Here, an avatar includes one or more objects, and the configuration of these objects can be determined relative to the immersion point of the first person of the avatar for depiction. As another example, the configuration of these and other objects and the avatar can be determined based on one or more markers located in the virtual environment, or based on some other mechanism, without departing from the principle of the present invention.

In FIG. 6, the gaze vector 205 combined with the first avatar 200 is parallel to the gaze vector 207 combined with the second avatar 202. However, it can be confirmed that such vectors 205 and 207 need not be parallel. For example, the vectors 205 and 207 may be asymmetrical. More generally, these vectors do not need to have a specific relative direction when combined in a mutual gaze.

It can be confirmed that although the virtual environment 201 in FIGS. 5 and 6 is illustrated as a rectangular three-dimensional space, the environment 201 is not limited to a rectangular space, for example, it may be a spherical or other space. Furthermore, the environment 201 can support more or less than three degrees of space without departing from the principle of the present invention. That is, among the spatial parameters suitable for the virtual environment 201, the present invention provides an interaction focused on attracting attention (as described below, regardless of whether the invention is spontaneous, planned, or a combination of the two and/or other Type).

It can also be confirmed that the virtual environment 201 in FIG. 5 and FIG. 6 is generally a space exceeding the attention space in size. When an avatar/participant moves in the virtual environment (for example, combined with the movement of the first immersion point and/or the third immersion point), the apparent attention space also preferably moves. In this way, the direction of the attention cone and vector in the relative attention space is better combined with the immersion point. Such a combination is preferably a fixed relationship. As an example, the combination can provide a Cartesian coordinate system in which (i) the initial connection with a respective immersion point 103, and (ii) the connection between an axis and an attention vector, for example, the X axis Follow the visual vector 106. Therefore, although the attention cone of an attention space is preferably completed so as to have a selected direction relative to the respective sensing and focus balls, the attention cone can be activated to have relative to the virtual environment The dynamic direction and position of the camera.

Under appropriate conditions, when one end is gazing at an avatar object, or when gazing at each other with an avatar, a specific embodiment of the present invention provides that a participant is appropriately activated to have an interactive dialogue. One end or mutual gaze leads to interaction, which is sometimes referred to as "gaze situation" here.

Whether a gaze situation is satisfactory is determined by the selected calculation. These calculations can be commensurate with the differences in the interaction (for example, tolerances related to time and attention to the preset dimensions of spatial elements). For example, the interaction with a non-incarnation object will typically convey the preset information of at least one object to other objects, while the interaction with an incarnation will typically have one or more aspects relative to humans. Real interactions (for example, impromptu and/or social aspects). In this way, an interaction involving an object without an avatar may perform calculations different from the interaction of an avatar.

A specific example of a gaze situation is as follows, combined with FIG. 7.

FIG. 7 shows the intersection of a gaze cone 204 and a plane determined by the line AAand an immersion point 213 of an object B in the gaze cone. The staggered pattern includes multiple characteristics: (i) the vector of the gaze vector 205<u>AA'</u>Combined with the gaze cone 204, (ii) the line segments AP and AQ intersecting with the apex 210 of the gaze cone 204, (iii) the angle PAA' is determined by the vector<u>AP</u>and<u>AA'</u>The composition is marked by the three-dimensional arc represented by the symbol I. The angle is the rotation angle of the gaze cone. (iv) The vector connecting the vertex of the gaze cone 204 to the position of the object B<u>AB</u>(For example, with Bs first-person field of view, the configuration of B can be given by the relevant immersion point 213), and the (v) angle BAA' is determined by the vector<u>AB</u>and<u>AA'</u>The composition is marked by the three-dimensional arc represented by the symbol J. Because<u>AA'</u>It is collinear with the commensurate axis of the cone 204, so it can be understood that the above characteristics fully illustrate the staggering purpose of the gaze condition calculation.

Generally speaking, when the gaze space 150 is established, the gaze vector is determined<u>AA'</u>The length of the angle and the size of the angle PAA'. However, the frequency of such changes in length and size is predicted to be much smaller than the frequency calculated for this gaze condition. Therefore, each such length and size is preferably regarded as a constant in the gaze condition calculation.

The mutual gaze situation in the first person's field of vision is depicted in combination with the characteristics of FIG. 6 and FIG. 7. In the depicted situation, the gaze cone 204 is combined with the avatar 200. In addition, the object combined with the first person immersion point 213 includes the avatar 202, and the position of the avatar 202 can be combined with the avatar 212. In this way, the participant A stares at the avatar 202 through the avatar 200 and provides:

|<u>AB</u>|＜|<u>AA'</u>|and α,&lt;α,

Where 0°<u>＜</u>α<u>＜</u>180° and 0°<u>＜</u>β<u>＜</u>&lt;180°. The comparison of the angles (β&lt;α) can be changed to the comparison of their cosine values: cosβ&gt;cosα. Furthermore, |<u>A A'</u>| And cosα can be expressed as L_aAnd C_aTo be replaced, the value of these replacements is determined by the gaze cone 204, and here the subscript symbol "a" is related to the constant A (as mentioned earlier, L_aAnd C_aThe value of can be regarded as a constant in these calculations without departing from the principle of the present invention). In addition, u_aGaze vector<u>AA'</u>The unit vector of X.

Replace the cosine calculation on the left with the dot product of the vector, and use the constant C_aAfter replacing the cosine on the right, the second part of the gaze expression becomes:

[(<u>AB</u>u<u>a</u>)/|<u>AB</u>|]&gt;C_a

Here (a)<u>AB</u>@<u>u</u>_<u>a</u>Is a vector<u>AB</u>And unit vector<u>u</u>_<u>a</u>Dot product of (b)|<u>AB</u>| Is a vector<u>AB</u>The length of (c) the symbol'/' means division.

Combining the above cosine conditions with quantities again, derive the following overall definition of the unilateral gaze condition of "A gazes at B":

C_gab=[(|<u>AB</u>|&lt;L_a)and(<u>AB</u>@<u>u</u>_<u>a</u>&gt;｜<u>AB</u>|*C_a)]

Here the symbol'*' means multiplication. Similarly, the overall definition of the unilateral gaze of "B gazes at A" is:

C_gba=[(|<u>BA</u>|&lt;L_b)and(<u>BA</u>@<u>u</u>_<u>b</u>&gt;|<u>BA</u>|*C_b)]

Here L_bIs the length of Bs gaze vector 207, C_bIs the cosine of the half angle of the gaze cone 206 of avatar B, and<u>u</u>_<u>a</u>Gaze vector<u>BB'</u>End the unit vector of X.

If A and B stare at each other, then it is in line with the condition of staring at each other. In this way, the situation of gazing at each other is expressed as follows:

C_g=C_gabAnd C_gba

Although the foregoing calculation of the single side or mutual gaze condition is described in the context of the avatar-avatar gaze in the first person's field of view depicted in FIGS. 6 and 7, it is understandable that the calculation is suitable for other gaze conditions. For example, the calculation of the single-sided gaze condition is suitable for a participant to gaze at a non-incarnation object from the first or third person's field of view. Furthermore, the calculation of the mutual gaze situation is suitable for (i) a first participant gazes at the avatar of a second participant in the field of view of a third person, and the second participant gazes at the avatar of a second participant in the field of view of the first or third person. One gazes with a third person's field of vision, and (ii) other gaze conditions selected to support the virtual environment. In addition, if the control field of view is completed, the calculation will use the control field of view gaze cone and its related characteristics, as described earlier. It is also understandable that although the above calculation is directed to vision, the calculation can be extended to other sensing, whether it is similar to human sensing or not, without departing from the principle of the present invention.

When multiple objects ("candidates") satisfy the gaze conditions of the first or third person ("target") of a participant, they can interact in a group. However, if a candidate is better, then the candidate can be determined based on the attributes of the object and/or the selection rules defined by a participant. For example, when the gaze situation is satisfied between a goal and two candidates, one candidate is an avatar and the other candidate is a non-avatar object, and the context emphasizes social interaction, then the avatar is better The land is chosen to have interactive priority. As another example, when the gaze situation is satisfied with the above situation, but the text does not emphasize social interaction, then the avatar or the non-avatar object can be selected to have interaction priority. The selection is based on general attributes and (Or) rules. The latter example is depicted in a hockey game, in which the avatars of the participants are players, and the non-available objects include the rubber disc: (i) The rubber disc can be selected at the first opportunity, especially for shooting the net. , Ice competition or similar matters, and (ii) a player can be selected at the second time, especially for defense.

If the attributes/rules fail to identify a single, high-priority candidate, then other better mechanisms will be supported. Other mechanisms include, for example, providing random selection, enabling the participant to choose among candidates, and supporting multiple candidate selection. The priority setting control (discussed in the following programmatic interaction) can preferably be used to instruct the system 8 to provide attributes for calculation and to select among candidates.

If the group interaction is better, the target can use the attention request control (discussed in the planned interaction below) to select one group member at a time for paired interaction while still interacting with other group members. In this situation, the priority element can be set to generate the highest priority for the selected pair of partners, the next highest priority for other group members, and the lowest priority for non-avatar objects. Dominated by these priorities, the targets sensing of the paired partners signals (e.g., vitality and voice) is strengthened, and at the same time, it senses signals from other group members whose level is higher than that of non-partners. The signal level of the group members. The goal can also use attention end control (also discussed in the planned interaction below) to exclude an object from a group (for example, the avatar and the participants represented by the excluded avatar). As a result, different members of a group can sense different group members.

The gaze calculation described above can be extended to sense other objects in the virtual environment. Although the gaze situation is combined with the focal space, the sensing is combined with the sensing space. Therefore, one method of providing sensing calculation is to change the gaze calculation by replacing the focus space parameter with the sensing space parameter. In the specific case of vision, the sensing space parameter will be replaced by the visual space parameter.

<u>Watch out for drift</u>

The preferred embodiment of the present invention illustrates attention drift. Pay attention to drift considerations Pay attention to changes in parameters and engage in interactive conversations. (Whether from the perspective of the first or third person, the participant or object engaged in such a conversation with the participant is sometimes referred to as a "partner".)

Note that drift is combined with most activities in this virtual environment. Examples of such activities include: (1) Avatar A moves position in the virtual environment, this action and the final position are confined to the visual/gaze cone relative to the interaction with partner B; (2) Avatar A moves at least relative to Interact with partner B outside of the visual/gaze cone; (3) A and partner B are relatively static to each other, but A shifts its gaze vector, so partner B will no longer be located in a gaze cone of A relative to the interaction; (4) ) A moves its position relative to B in the virtual environment and drifts its vision/gaze cone, so that partner B will no longer be located in a relative gaze cone of A; (5) A with or without partner B, Its gaze cone intersects with a non-partner C's gaze cone; (6) the non-partner explicitly attempts to attract A's attention; and (7) the non-incarnation object sends a signal to attract A's attention.

In an attention drift situation regarding a virtual object moving within the attention cone of its partner, a possible specific embodiment is to ignore such movement when engaging in an interactive dialogue with the partner. However, it is preferable that such movement is a factor in the priority element 54 of the system 8. Under this factor, although the object moves, the object seems to be frozen in position to its partner during the interactive dialogue. However, below the appearance of the coefficient, a response to the movement is provided, increasing (for example, making it more natural) appearing in the virtual environment. For example, when the object moves closer to its partner, it provides more details and louder/clear sound of the object; and when the object leaves relative to its partner, it provides There is less detail on the object and a quieter/less unclear sound. (In this example, it is understandable that the term "relatively closer" is generally used to reflect the configuration of the virtual environment.)

In the condition of attention drift, where mutual gaze is broken, it is preferable to provide one or more limits of the tolerable coefficient of attention drift on the gaze situation. The noticeable drift tolerance coefficient is preferably temporary and combined with predetermined system activity. Examples include: (i) If avatar A moves outside the relative gaze space of B during an interactive dialogue, the system will warn the participant who is combined with avatar A, for example, send a text warning message to the participant The host of the participant, and (ii) if the avatar A is still moving outside the relative gaze space of B, beyond the selected allowable time, then the system will infer that the participant of the avatar A is not interested in continuing the conversation with B. At this time The system will take a predetermined action, for example, to end the conversation. (It can be confirmed that ending the conversation based on exceeding the allowable time is related to establishing the conversation based on keeping gaze longer than the selected time.)

In completing the tolerable coefficient of attention drift, the system 8 preferably tracks the influence of the coefficient on the gaze condition value. For example, in the context of the temporary coefficient, the system tracks the time that the gaze condition has maintained or has maintained a general value, and compares the tracked time with the selected allowable time. In a specific embodiment, the tracking method causes the calculation of the gaze condition to be modified as follows:

Let C_gIs the value of the mutual gaze condition, then C_g=C_gabAnd C_gba；

Let t_gIs C_gThe time for which a general value has been maintained or has been maintained;

Let T be the allowable time to pay attention to drift;

therefore:

If the following conditions are true: (C_G= True) and (t_g&gt;T), then A and B can have a spontaneous interactive dialogue; and

If the following conditions are true, a spontaneous interactive dialogue between A and B can automatically end: (C_g= Pseudo) and (t_g&gt;T).

It can be confirmed that the gaze of the single side can be used for C in the above calculation_g, In order to maintain the application and (or) content development needs. It can be confirmed that the allowable time T does not need to maintain the same value in all situations. For example, in the above equation, the allowable time T can hold different values for the initiation, implementation, and end of one or more conversations. As another example, the value of T may be based on the behavior history of a participant in the virtual environment. A description of the latter example is to let the value of T be a function of the frequency of attention drift: for example, if the participant is restricted to paying attention to drift quite frequently (relative to a predetermined value, or relative to an environmental statistical or dynamic Statistical characteristics, or other measurements), T increases when initiated and (or) decreases when it ends, or vice versa, or a combination.

<u>Clear control</u>

A planned interactive dialogue is through the better choice of explicit control. In addition, however, explicit control may also be related to spontaneous interaction. In any situation, the explicit control can be used by a participant, an avatar (e.g., typically based on a participant's affirmative activity), or a non-incarnation object (e.g., based on a program). Understandably, explicit control can be used for the first or third person's field of vision. As another example, a planned interaction can be initiated by a partner, even if the partner is placed outside the attention space of an avatar, because the participants of the avatar can preferably be based on the general third persons vision or based on the partners Use clear control over identity.

The explicit control preferably includes the following:

Display note: A control trigger displays the sensing and/or focus spaces 56, 58. Generally speaking, the sensing and focus spaces 56, 58 are preferably not displayed, however, the space can be displayed by activating one or more display attention controls. (An example of this control is a mouse click on the face 110 of an avatar.) When activated in this way, the control preferably causes the associated space 56, 58 to be graphically (i.e., like a cone), text (i.e., , Displayed as the configuration parameters of the space and the state of the object) or a combination of the two. It can be confirmed that the related space can be displayed in other ways without departing from the theory of the present invention.

Hidden attention: a control that makes the displayed sensing and/or focus spaces 56, 58 disappear. This control can be accomplished in different ways, including: (i) the display attention control box; (ii) clicking on a display gaze cone 152; and (iii) a text style (for example, a menu).

Setting Note: Establish a control for one or more sets of parameters related to the sensing and/or focus spaces 56, 58. A set of parameters is preferably related to the size of the individual space, for example, the coverage angle of the individual vision/gaze cone 102, 152 (angle PAA' in FIG. 7) and/or the individual vision/gaze vector 106, 154 length. The predetermined length of a sensing vector is preferably the radius of the individual sensing ball 104, and the predetermined length of a focus vector is preferably the length of the individual sensing vector 106. The various sets of parameters can be used for different purposes, including: (a) controlling attention drift, for example, the allowable time for attention drift, and (b) expressing or responding to a participants sociality at any particular point in time or period of time . As an example of the latter, a participant sets a gaze space 154 for (i) a smaller value that affects the departure, or (ii) a larger value that affects the social interaction. This control can be done in different ways, including through a function menu and through the direct manipulation of a cone and/or vector icon (for example, in the case of a non-incarnation object, responding to a selected logic-based or based Rule-based decision).

Steering Note: A control for turning a sensing/focus vector and cone steering. In the first and third persons field of vision, the direction of the sensing/focus vector is preferably controlled by the direction of a related object (for example, the direction the participants avatar faces, or the direction of the third persons camera). Direction) control. In this situation, a control separated from the self-determination of the object direction does not need to complete the steering attention control. However, an individual control can be provided without departing from the principles of the present invention. In the first person's field of vision, if the direction of a vector can reach a surface of the avatar where the immersion point is arranged in addition to normal, then this type of control is better. However, with a third-person view, a control will typically be combined with the control of the immersion point. To control the field of view, a control is preferably linked to the attention cone of the control field of view. (Because this control controls the direction of the attention cone, it can be confirmed that this control is an example of a clear control of the dialogue between planned and spontaneous interaction.)

Attention requirement: A control that requires attention. In response to the activation of the control, an attention request message is automatically sent to the prospective partner. As an example and on the premise of completion, if the control is activated, then all or part of the following are available: (1) A related sensing/focus vector and the activation source (for example, a participant or a (Objects) combined to become directly pointing to the prospective partner, (2) a related sensing/focus cone combined with the priming source, moved closer to the prospective partner (if the partner was originally outside the cone); 3) The detailed information about the startup source is ready to be sent to the prospective partner; and (4) The startup source (for example, via the host) is ready to receive and/or start working (for example, display) details about the prospective partner News. This control can be done in various ways, including a definite mechanism (for example, using a pointing device such as a mouse to click twice on a selected object feature such as a face) and/or via an automatic The mechanism of (e.g., responding to a chosen logic-based or rule-based decision in the absence of an incarnation object). Furthermore, completion preferably provides that the prospective partner receives the attention request message to trigger an automatic and responsive activity. For example, in a situation where the prospective partner has a related participant, the partner receives the attention request message, and preferably displays an attention request signal on the host of the partner, for example, on the partners participation The image used by the person on the screen. Examples of the completion of this kind of attention request signal include: (i) flashing otherwise adjust the image (or part of it) of the object combined with the attention request message, (ii) the image (or part of it) combined with the attention request message The object provides a selected gesture, expression, or other signal (for example, smiling or waving), and (iii) displaying a selected message (for example, displaying a text on the video screen used by the partners participant) message).

Attention agree: agree to a control of attention. When this control is activated in response to an attention request, the prospective partner sends an attention consent message to the attention request source. As an example and on the premise of completion, if the control is activated, then all or part of the following are available: (1) A related sensing/focus vector of the prospective partner becomes directly directed to the source of the attention request; (2) A related sensing/focus cone combined with the prospective partner moves closer to the attention request source (if the attention request source is originally placed outside the cone); (3) Information about the prospective partner The detailed information is sent to the attention request source; and (4) the prospective partner (for example, via the host) is ready to receive and/or start to engage in (for example, display) detailed information about the attention request source. When the attention request source receives (for example, via the host) the attention consent message, the attention request source and the partner can conduct an interactive dialogue. This control can be done in various ways, including a definite mechanism (for example, using a pointing device such as a mouse to hit the "y" key on the object of the attention request source, or via a pointing device on the attention request signal Click once on the icon of) and/or through an automatic mechanism (for example, in the case of an object without an avatar, responding to a selected decision based on reverie or rule). (The following is also preferable. In a situation where a participant initiates a planned interaction with an object, the interactive dialogue is thus combined with the enhanced provision and/or related of the object on the host of the participant The display of extended information about the object, for example, on its webpage.)

Non-attentional consent: a control that rejects an interactive request. When this control is activated in response to an attention request, the prospective partner sends a non-attention consent message to the attention request source. The completion of this control preferably provides that the attention request source receives the non-attention consent message to trigger an automatic and responsive activity. For example, if the attention request source is combined with a participant, then an unattentional consent signal is preferably displayed on the host of the attention request source, for example, displayed on the participant's video screen. Examples of the completion of such non-attentional consent signals include: (i) making all or part of the object fade away from the attention-seeking place (for example, removing the face of an avatar), (ii) providing the object with a selected posture, Expressions or other signals (for example, frowning, shaking the avatars head, or preventing a wave of hands) and (iii) providing a text or other message appearing at the source of the attention request (for example, showing a text message on the participants image On the screen), in any case, when the attention request source receives the non-attention consent message (for example, on a host), it is preferably activated to choose whether to stop processing the detailed information about the prospective partner. This control can be done in various ways, including a definite mechanism (for example, using a pointing device such as a mouse to hit the "n" key on the object of the attention request source, or on the icon of the attention request signal Click twice) and/or through an automatic mechanism (for example, responding to a chosen logic-based or rule-based decision in the absence of an incarnation object).

Note the end: a control to end the interactive dialogue. When the attention request source (partner) starts this control, an attention end message is sent to the partner (attention request source). As an example and on the premise of completion, if the control is activated, then all or part of the following are available: (1) The sender of the message stops sending detailed information to the recipient of the message; (2) The sender stops processing the receivers detailed information; and (3) the receiver stops sending detailed information to the sender, stops processing such information for the sender, and/or stops processing received from the sender Information. This control can be accomplished in various ways, including a definite mechanism (for example, using a pointing device such as a mouse to hit the "e" key on the object of the attention request source) and/or via an automatic mechanism ( For example, it is like responding to a chosen logic-based or rule-based decision in a situation where there is no incarnation object).

Do not disturb: When a participant/object interacts with a partner, a control that suppresses the attention request message. Preferably, even if the control is set, when the recipient is not interacting, the attention request message will not be suppressed. Furthermore, the control is preferably completed so as to be resettable (for example, when an interaction is terminated or the related permissible disturbance control is set, the original do not disturb control can be automatically reset so as not to suppress the attention request message). This control can be done in a variety of ways, including a definite mechanism (for example, via a button or an input in the function menu) and/or via an automatic mechanism (for example, in the case of a non-incarnation object, Respond to a chosen logic-based or rule-based decision).

Disturbance allowed: When a participant/object interacts with a partner, a control that does not inhibit the attention request message. This control can be completed in various ways, including when resetting (setting) the do not disturb control, providing a box for setting (resetting) this control.

No attention drift: When a participant/object interacts with a partner, it is forbidden to pay attention to a control of drift. As an example, this control can be supported by setting the appropriate allowable time to 0. This control can be done in various ways, including as described below or with a button or an input in the function menu.

Allow attention to drift: A control that allows attention to drift. When this control is set, whether it is busy interacting or in leisure, an interactive dialogue can preferably be provided through mutual gaze or explicit control or both. This control can be completed in various ways, including providing a block to set (reset) this control when resetting (setting) the inadvertent drift control.

Display attributes: display a control for the attributes of the selected object. In the case of an avatar, among other information, the content of an attribute preferably includes the relevant participants likes, dislikes, hobbies, and personality. This control can be done in various ways, including using a pointing device (such as a mouse) to hit the "e" key on the object.

Set attributes: Start a control that changes the content of an object's attributes. An attribute can be configured to provide and/or supply a variety of different versions or parts that are accessible. For example, if an attribute is provided to an object without incarnation, the attribute provided can be selected according to the type of the object. As another example, if an attribute is provided to an avatar, the attributes provided can be selected according to the related participants of the avatar, or by the types of groups and/or participants. To illustrate the latter example, participants who appear in a friend list combined with this attribute can receive a more detailed version than those who are not in the list, and furthermore, participants who appear in a list of haters May be completely restricted from receiving or entering an attribute. This control can be done in a variety of ways, including providing forms to display or change attributes.

Set priority: Set a control for calculating priority parameters. Regardless of the first or third person's field of vision, a participant can preferably selectively change the attributes of the objects passing by in the virtual environment. Examples of such objects include avatars (including the participants avatar, especially the visible part) and non-avatar objects (including the part of an avatar). To illustrate, this control allows the participant to increase the priority of the voice signal of a friend's avatar, so that when the friend enters the virtual environment, the participant can clearly hear the friend's voice signal. To further illustrate, this control allows the participant to lower the priority of a non-partner object that is playing an unwelcome message. Preferably, the participant can also configure the system through this control so as to resolve conflicts with a number of objects that attract the participant's attention. This control can be done in various ways, including providing forms and sliding keys.

The use of explicit controls including the above initiates an automatic interactive dialogue. In the context of two avatars, an example of the processing of an interactive dialogue includes: (1) a first participant (a) uses the steering attention control to indicate a focus cone towards the avatar 204 of a second participant, or (b ) Check a related focus cone of the second participant, for example, through the display attention control, in order to determine that the second participant can participate in the interaction, that is, it has not interacted with a partner; (2) Use display attribute control , The first participant checks the attributes of the second participant to determine that the second participant meets the needs; (3) the first participant requires the second participants attention by activating the attention request control, The first participant waits for the second participants response accordingly; (4) After receiving the first participants attention request message, the second participant checks the display that the first participant seeks attention. Attention request signal; (5) The second participant uses the display attribute control to check the attributes of the first participant; (6) The second participant determines whether the first participant is suitable for the needs based on the above-mentioned inspection or other factors (7) The second participant (a) activates the attentional consent control, or (b) or activates the non-attentional consent control, or (c) uses the steering attention control to turn its associated gaze cone 206 away from the first Participants avatar, or (d) relying on logic/rules to automatically determine the response; (8) If the first participant receives the second participants attention and consent message, then the first and second participants will be activated (9) If the first participant receives the second participants inadvertent consent message or detects the turning of the gaze cone, then the interaction between the first and second participants is not activated. When any participant activates the attention end control, the interactive dialogue is terminated.

Although the above series of steps are carried out, it can be confirmed that other series are also suitable for interactive dialogue. For example, some of these steps can be omitted. As another specific example, one or more of steps 1, 2, 5, and 6 can be omitted, especially when the two participants are similar to each other. It can also be confirmed that the remaining explicit controls, including other explicit controls described previously, can be used in the above or other series without departing from the principle of the present invention. It can also be confirmed that although the explicit control can be described as activated by an object, when the object is an avatar, the activation is preferably active or inactive (ie, one or more such controls may be It is automatically activated and completed, regardless of the control, the activation of a participant will also be completed).

<u>Interaction between planning and spontaneity</u>

Figure 8 depicts an exemplary state transmission interweaving programmatic and spontaneous interactions, preferably in combination with an interactive control engine. In this state transfer, a participant and its avatar are represented by A, an initial partner is represented by K, and an intruded participant/avatar/object ("intruder") is represented by F. An intruder initiates an attention request control, and thus sends an attention request message to A, when A is in any state other than the basic state. It is understandable that if A is in a basic state, then it is not an interactive dialogue.

The state and symbols of the interweaving interaction include:

B: Basically. A does not involve any objects.

Wr: Wait for the remote end to agree. A is waiting for a prospective partner to agree to pay attention.

Ws: Waiting for local agreement. A is waiting for consent and attention to a prospective partner.

G: Staring. The gaze between A and a prospective partner is real.

I: Interactive. A is interacting with a partner.

P: Farewell. When A is still interacting with a partner, his attention drifts.

FG: f-gaze. When A interacts with a partner, the gaze between A and an intruder is real. D: Excuse me. When A interacts with a partner or a prospective partner, he receives an attention request signal from an intruder.

The fields and symbols of an interactive data structure include:

Δt: The period during which A is in a special state and interacts with a partner or an expected partner at the same time.

Δt_new: A is placed in a special state and is disturbed by an intruder at the same time, or the gaze between A and an intruder is a real period.

S: A's current state.

S_old: Save to keep this state, and state A has made/has made a transition.

Partner: Save to keep the current partner's identification.

Intruder: Store to keep the identity of the current intruder.

In Figure 8, circles indicate states, and arrows indicate transitions between states. The mark in a circle consists of two parts: the state symbol and the state action. The mark on an arrow consists of two parts: the condition that triggered the transition and the action performed in a transition. The actions of the individual conditions and transitions are enclosed in square brackets to indicate which action should be performed when the transition is in progress. The symbols used for this depiction of states and transition actions include:

++: Increases in units of preset time intervals (for example, per second, milliseconds, microseconds, etc.).

op1: Partner=Intruder=NULL; Δt=Δt_new=0; S_old= Illegal (ILLEGAL).

op2: partner=KID, where KID is the identification symbol of an expected partner. op3: AEs; partner=FID, where FID is an identification symbol of an expected intruder. Δt=Δt_new=0；

Intruder=NULL; S_old= Illegal (ILLEGAL).

op4, S=S_old;Δt_new=0; Intruder=NULL; S_old= Illegal (ILLEGAL).

op5: S_old=S; intruder=FID; Δt_new=0。

op6: Δt=Δt_new=0; Intruder=NULL; S_old= Illegal (ILLEGAL).

op7: s_old=Illegal (ILLEGAL); Δt=Δt_new=0; intruder=NULL.

op8: AEs; partner = intruder; intruder = empty (NULL); Δt = Δt_new=0; S_old=Illegal (ILLEGAL); the symbols used to depict trigger conditions include:

AEs: A has sent the end of attention to this partner.

AEr: A has received attention from this partner.

AEsr: A has sent the end of attention to this partner, or has received the end of attention from this partner.

AGs: A has sent an attention agreement to a prospective partner.

AGr: A has received attention and consent from a prospective partner.

ARs: A has sent an attention request to a prospective partner.

ARr: A has received a request for attention from a prospective partner.

C_g: The value of the gaze between A and a partner or prospective partner.

Di: When A interacts with a partner or prospective partner, allow interruptions from an intruder. Dr: A's attention is activated to drift.

FAEsr: A has sent the end of attention to an intruder, or has received an end of attention from an intruder.

FAGs: A has sent an attention agreement to an intruder.

FNAGs: A has sent an unintentional consent to an intruder.

FARs: A has sent an attention request to an intruder.

FARr: A has received an attention request from an intruder.

FC_g: The value of the gaze between A and an intruder.

NAGs: A has sent a non-attentional consent to a prospective partner.

NAGr: A has received inadvertent consent from a prospective partner.

T_g: Allowable time from staring to the start of interaction.

T_P: Separation to allowable time to be regarded as attention drift.

T_W: Allowable time of Wr and Ws state before the suspension.

T_d: Allowable time to state D or state FG for the longest period.

&amp;: logical AND operation.

|: Logical OR (OR) operation.

It is understandable that developers can configure their respective systems regarding the relationship between the allowable time to complete the interweaving interaction. To give an example, the T_w&gt;T_gThe relationship (and the allowable value set therein) can be selected to provide a longer time than the spontaneous interaction to initiate the planned interaction. To give another example, the T_P&gt;T_g&gt;T_dThe relationship (and the allowable value set therein) can be selected to provide that when a partner interacts, the selected participant/object has the partners attention. This allowable relationship requires an intruders determined action or Increase the gaze time to disengage from the partner and interact with the intruder. The selected configuration generally corresponds to different elements, including the details of available system resources and application areas.

When A is in the basic state B, A has not conducted an interactive dialogue. When it sends a attention request to K, A enters the state Wr, waiting for K to agree to transfer the attention to A. If A has received a request for attention from K, A enters the state Ws, waiting for agreement (or disagreement) to transfer the attention to K. In either case, the transition operation op2 is executed.

When an interaction trigger signal (for example, state Ws, AGs, or ARs; state Wr, AGr, or ARr) arrives, A enters the interactive state and interacts with K. Among them, the conversion operation op6 is executed. However, if a suitable signal does not arrive within the predetermined time or if A sends an unattentional consent signal to K, or receives an unattentional consent signal from K; or if A sends an attention end signal to K, Or receive the attention end signal from K; A will return to the basic state B. Among them, the transformation operation op1 is executed.

A can enter the gaze state G from the basic state B, so that the proper gaze situation between A and K becomes real. Among them, the transformation operation op2 is executed. If the proper gaze condition lasts for at least the preset allowable time (e.g., Δt&gt;T_g), then A enters the interactive state. Among them, the conversion operation op6 is executed. If A sends an attention end signal to the current partner (such as K), or receives an attention end signal from the current partner (such as K), then any of the interaction and gaze states (G and I) will change Return to state B.

If (i) A is subject to watchful drift, and (ii) condition [(C_g=false) and (Δt&gt;T_g)] is true, then it will also return to state B from the gaze state G. Among them, the transformation operation op1 is executed.

From the gaze state G, if A receives an attention request from the prospective partner K, then A can also transition back to the state Ws. In this transition, the transition operation op2 is executed, so that the partner stores the identification code of the object that keeps sending the attention request. It can be confirmed that although not shown in FIG. 8, when A is in the state Ws, the interactive control engine 39 can be completed to automatically start the attention and consent control that replaces A. In this situation, [(C_g= True) and (Δt&gt;T_g)" becomes real.

If (a) A is configured to allow attention drift, (b) A and/or As current partners attention is drifting, and (c) A and/or As current partners no longer meet the gaze condition, then A It is also possible to change from the interactive state to the separated state P. If (i) the gaze state is re-established and (or) (ii) the drifting A (or B) issues an attention consent or attention request to the current partner (or to A), then A can be in the separated state P, Re-enter the interactive state with A's current partner. In a completion, if the related control can end at the selected allowable time (for example, the separation allowable time T_p, It is released from the time of entry to the state P), then the system can provide re-entry through these clearly controlled releases. However, if (a) A or As current partner issues an attention end, or (b) the attention drift exceeds the allowable time (for example, the situation [(C_g= Pseudo) and (Δt_new&gt;T_p)] is true), then the interaction ends, and A enters the basic state B. In this transition, transition operation op1 is executed.

The disturbing state D illustrates the following situation. When A is in a state other than the basic state, and A is configured to be disturbed, an intruder (for example, F) sends an attention request to A. As depicted in Figure 8, if the intruder F sends an attention request to A, and A is busy (for example, in state I, D, G, or P), then A changes to the disturbed state. Among them, the transformation operation op5 is executed.

If A issues an attention agreement to the intruder (for example, F), then the end of an attention is sent to the current and previous partner (for example, K) in the form of the conversion operation op8. In this situation, A changes to the interactive state, and the current-previous partner becomes the current-previous intruder (for example, F).

In the disturbing state D, if A issues an inadvertent consent or an attention end to the intruder; or if A receives an attention end from the intruder; or if A issues an attention consent or an attention request to the intruder Partner (for example, to K); or the end of the allowable time during which A can stay in state D (for example, Δt_new&gt;T_d); Then A returns to the state before it changed to the disturbing state.

Preferably, A can be transformed into an f-gaze state (FG) under different conditions. For example, FIG. 8 illustrates a specific embodiment in which A can transition from state Ws, Wr, G, FG, or P to state FG. In these transitions, A is configured to allow attention to drift (for example, Dr is real), and furthermore, if the gaze between A and the intruder is real, then the transition can occur. In each situation, the transition operation op5 is executed.

If (i) the gaze between A and the intruder was real at least during a preset allowable time (for example, Δt_new&gt;T_g), or (ii) A sends an attention consent to the intruder, then A can change from the FG state to the interactive state. As in the meantime, the conversion operation op8 is executed, so that the partner stores and maintains the identification code of the intruder, and the intruder store becomes NULL. In the transition, a notice end is also sent to the current and previous partner, for example, to K.

If the following conditions occur, then A will return to the state before it transitions to the FG state; (i) the gaze between A and the current partner (such as K) becomes real, and (ii) the relevant A can stay in The allowable time during the state FG ends (for example, Δt_new&gt;T_d), (iii) A issues (for example, to F) an inadvertent consent or an attention end to the intruder, (iv) A receives an attention end from the intruder (for example F), or (v) A releases An attention consent or an attention request is given to the current partner (for example, to K). If the initial state in which A is disturbed is the interactive state, then preferably the interaction with the partner can continue, even if A is now in the state FG. In the transition back to the initial state, the transition operation op4 is executed.

If A issues a notice request to an object other than the current partner (for example, other than K), then A can transition from the state other than the basic state B to the state Wr in order to wait for the prospective new partner to agree/ Refuse to pay attention. In this transition, the transition operation op3 is executed.

When A is in the standby state Ws or Wr, or in the state FG, it is better to ignore an attention request from an intruder in order to minimize the complexity of completing the state transition. As for other states, explicit controls that should not be disturbed (see above for spontaneous interaction) can be used to prevent an intruder's attention request.

In any situation, the explicit control of the unattended drift can be used to prohibit A's attention drift.

<u>Priority component</u>

An interactive dialogue is better given priority in processing. As depicted in FIG. 1, the system 8 includes the priority element 54 to give priority to processing. Generally speaking, the priority element 54 individually provides each participant with the priority of objects in the participant's attention space. This priority is better used not only to determine which object will be processed, but also to determine the details of execution. This priority is also preferably used to resolve conflicts about multiple objects seeking attention. In addition, the priority is also preferably used to determine which object will be sent via the network and its quality level. After the priority is adopted, the resources of the system are generally used to be more efficient. According to the present invention, the advantages of priority-based processing and the arrangement of data communication in any situation include the coordination of the priority and attention components.

The priority is preferably used to provide processing/communication enhancement details and/or those objects deemed important by the participant (or developer). For example, the object that is interacting with a participant preferably has a high priority, and at the same time can be provided with one or more senses in itself, so that the participant can experience enhanced performance of the object's characteristics. In an attention space with a vision-based model, the details typically include the objects characteristics, expression, posture, and other manifestations; for its part, the improvement refers to one or more resolutions (for example, multiple polygons). ), enhanced mobility, and increased lens rate. In an attention space with auditory as the main model, the details typically include the sound of the object, and each sound is typically the following alone or in combination with the voice, music, or other heard sounds made by the participant; On its own, the increase refers to one or more of the volume, resolution, increased bandwidth, and reduction of the mask of other sounds (for example, in order to do so, the sound must be louder and clearer, otherwise it will not be able to Uninteracted objects are distinguished).

In a specific embodiment, the objects in a virtual environment include source objects and media objects. Source objects are objects that can interact with a participant, such as avatars, dogs, and televisions. Intermediate objects refer to objects that simulate a participant's senses when processed, such as activity, images, sounds, text, and pictures. Typically, more than one intermediary object is combined with a single source object. (Therefore, a source object in this document sometimes refers to the source of the related media object.) As described below, when a single attention space is used in the calculation of priority, the basic priority of a media object is higher. The best place is determined by the relationship between its source object and the attention space used for the calculation.

The priority is better judged to be related to the selected parameters. Typical parameters include one or more of the following: (i) the position of an object relative to the attention space selected by one or more participants, (ii) the direction, orientation and length of the selected sensing and focal space, (iii) The attributes of the participant, (iv) the parameters explicitly set by a developer or participant for the priority calculation, (v) the social interaction/cultural norms model, and (vi) the economic model, for example, consumption structure.

Among the selected parameters, one or more parameters are preferably used as basic parameters, while other parameters are used as correction parameters. In this situation, the priority is first calculated using the basic parameter. The priority is corrected by calculation using one or more of the correction parameters. In the discussion of the following explanation, the background is the first persons field of vision and the two basic parameters explained: (a) about (i) the participants avatar A and (ii) the attention that exists in combination with the avatar A A basic parameter of the relative distance between another object B in space, A and (b) the vector combined with the relative position of (i) and objects A and B<u>AB</u>And (ii) Another basic parameter of the angle between an attention vector combined with A. Although these basic parameters are preferably located in the background of the first person, it can be confirmed that other basic parameters can be selected and translated without departing from the principle of the present invention. It can also be confirmed that the background may be other than the first person's field of vision, without departing from the principle of the present invention. For example, in the background of the third persons field of vision, (a) a basic parameter can be selected as (i) the immersion point of participant A and (ii) another object B that exists in the attention space combined with A The relative distance between the two, and (b) Another basic parameter can be selected as a vector that combines (i) and the relative position of the immersion point A and the object B<u>AB</u>And (ii) the angle between an attention vector combined with A.

Priority-based processing and data communication considerations use multiple attention spaces, each of which is combined with a specific sense (for example, sight, hearing, or smell). In this situation, processing and communication can have their own priority structure for each sensing. For example, although vision preferably has a priority structure related to the basic parameter of a visual space, hearing also preferably has a priority structure related to the basic parameter of an auditory space, wherein the visual space The basic parameter may be the same or different from the basic parameter of the visual space.

Priority-based processing and data communication are also considered to support multiple sensing at the same time, and each sensing has one or more attention spaces associated with it. In this support, the intermediary object of a specific type and combined with a single source object preferably has the same priority as the priority of the source object. For example, it is better to calculate the priority of a selected translation of the video space, and calculate the priority of graphics, video, and image objects, and if the source of the object is a single incarnation, then the priority is equivalent to that of the location. The priority of this avatar for which the selected video is available. To give another example, it is better to calculate the priority of a selected auditory space, and if the source of the object is a single incarnation, then the priority is equivalent to the selected auditory space The priority of this incarnation of space.

1. Use basic parameters for priority calculation

As mentioned above, with basic parameters selected according to distance and angle, the priority calculation of each existing object B includes (i) dividing A's suitable attention space 55 (see Figure 1b) into multiple regions, and (ii) Assign a number to the existing object B according to the area where each object of this type is located. The designation preferably follows the selected conditions, including: (a) The priority assigned to the area closer to A (for example, the apex of the attention cone closer to A) is assigned higher priority than the area farther away from A , And (b) configure the area of the attention vector closer to A (for example, the visual vector<u>AA"</u>) The assigned priority is higher than the priority of the area farther away from the attention vector of A. As an example, the configuration of an existing object B relative to A is based on the distance |<u>AB</u>| To measure, and its relative to<u>AA"</u>The angle of is measured at angle J (angle BAA").

Referring to Figure 1b, each supported attention space is preferably a ball 57. In order to provide an area, the suitable space (or a part selected accordingly) is divided along two ranges: (i) one of the spheres 57 of the attention space, which is related to the one or more radii of the concentric spheres The radius range (R-range) and (ii) an angle range (β-range) related to an angle β (angle BAA"). In each range, the sphere of the attention space can be selected to be equal, substantially equal, or Divide with unequal intervals. Moreover, in this division, the size of the area is controlled by selecting the value of the division interval related to each field. As a result, generally choose a value that is smaller than the division interval. It is provided to a smaller area. On the other hand, the selected translation is generally provided to a larger area with a larger value relative to the division interval.

In establishing this area, integer or floating point arithmetic can be used. It is understandable that the use of integer operations provides relatively less computational load, while the use of floating point operations provides relatively more precise calculations.

Examples of split patterns in the R- and β-domains include:

(a) Equal division in the R-field, the division radius increases equally from 0 to the radius R of a selected attention sphere (for example, i*R/n, where i=0, 1, 2, ... .n); (hereinafter sometimes expressed as R-division) (b) unequal division in the R-field, the division radius increases unequally from 0 to the radius R of a selected attention sphere (for example, According to SQRT[i*R²/n], where i=0,1,2,...n); (hereinafter sometimes referred to as R²-Divided representation)

(c) In the equal division in the β-domain, the internal angle of rotation of the division cone increases equally from 0 to 180 degrees across a selected attention sphere (for example, i*180°/n, where i=0 ,1,2,...n); (hereinafter sometimes expressed as β-partition)

(d) Unequal division in the β-domain, where the internal angle of rotation of the division cone increases unequally from 0 to 1 across a selected attention sphere (for example, i*2/n, where i=0 ,1,2,...n); (hereinafter sometimes expressed as cosβ-division) other division patterns can be used without departing from the principle of the present invention.

Choosing the partition pattern for each field leads to different combinations. The four possible combinations (including those based on other than examples) are described as follows: (i) the combination of R-partition and β-partition; (ii) R²-Combination of partition and cosβ-partition; (iii) Combination of R-partition and cosβ-partition; and (iv) R²-Combination of division and β-division. Understandably, for sight and hearing, R²The combination of -partition and β-partition is generally closer to the real-world behavior. It is also understandable that the division of a single field can be used for the calculation of the priority without departing from the principle of the present invention.

Each combination provides a priority number to the object in A's attention space. Among the priority numbers so provided, each combination preferably produces three basic priority structures. The first structure includes the priority number assigned in the R-field, for example using R- or R²-Division. The second structure includes the priority number assigned in the β-domain, such as using β- or cosβ-division. The third structure involves mixing priority numbers in the two fields. (If the priority number is calculated only by the basic parameter, then a structure is a basic priority structure. If the priority number is calculated by the basic parameter and one or more of the modification parameters, then a structure is a modification Priority structure.)

The general form of a priority structure is

{Division type (A_i, P_i)*｝

Where A_iIs the i-th object recognition symbol; P_iRefers to A_iUse the priority number of the displayed division type; and the'*' means zero or more pairs (A_i, P_i). Valid partition types include, for example, R, R2, B, C, RB, RCR2C, and R2B, where R represents R-partition, and R2 represents R²-Division, B stands for β-division, C stands for cosβ-division. The first four valid division type display priority numbers are generated by dividing the sensor ball along a field using only the displayed division type. The last four effective division types display the priority numbers of the mixing are generated by dividing the sensor ball along two areas using the displayed relevant division types.

Preferably, a smaller priority number has a higher processing priority. However, it is understandable that a higher processing priority can be combined with a larger priority number. Generally speaking, the priority number does not even need to be a number; rather, without departing from the theory of the present invention, any selected code representing the priority order in the attention space (for example, by The number P in the sexual structure_iMapping to the selected sequence system) can be used.

It is understandable that developers can configure their individual systems for priority calculations. For example, the developer can choose (i) one of the above-mentioned division types, or other division types, for priority calculation, (ii) all or part of the three priority structures, and (iii) which is used to complete the group The parameters of the sum and priority structure, as well as the related calculations. The selected configuration generally corresponds to different factors, including the available system resources and the characteristics of the application range (for example, the importance of the evaluation of each priority structure to control the processing of objects in different media, or the same media Different aspects).

2. Use other parameters to modify the priority

As explained above, the basic parameters are preferably used to establish one or more basic priority structures that connect a priority number to each individual object in the attention space suitable for the participant. The priority number is so calculated that it is easy to modify. The modification calculation is based on one or more modification parameters. One such parameter is an attribute of a participant, which, in terms of general sensing, includes a list of one or more points of views and insights of the participant in a specific category. Examples that can provide this list include: friends, bad guys, public characteristics, and interests. Other modification parameters can be economically related and related to whether the participant is engaged in an interactive dialogue. For example, if A and object B are engaged in an interactive dialogue, then Bs processing priority is P_BIt is preferable that the participant pays attention to the highest among all objects in the space.

<u>Attention and priority components applicable to objects</u>

As previously described, the attention and priority elements of the system are applicable to avatars and non-avatars. This section provides other examples and specific embodiments of the avatar object in the virtual environment.

1. Spontaneous interaction with objects

In addition to activating spontaneous interaction with the avatar, the mechanism presented in the present invention can also activate a participant to spontaneously interact with an object without an avatar. In other words, the system can be implemented to provide a sufficiently continuous focus (eg, gaze) that is related to an object triggering one or more defined actions that the object is performing. For example, when an object stares at a door for enough time (for example, a preset trigger time), regardless of whether it is in the first or third persons field of view, if the system supports the following processing and defines it as the doors Behavior, Na Mo can see that the door is open and the object is located next to the door. If the object is a dog, it can bark and wag its tail happily. As another example, if a participant stares at a picture for enough time (for example, a preset trigger time), and the picture is a link, the content (for example, a web page) is obtained and displayed.

The system can also be implemented to provide an interruption of focus (eg, gaze), the focus is related to an object triggering one or more defined behaviors of the object to stop. As an example, an interruption in gaze can trigger the door to close, and the dog barks in a sad tone with a drooping tail.

The system can also be completed to initiate different behaviors based on different focus durations. For example, if you stare at the dog for a long enough time, the dog will start rolling and jumping.

The above tasks can be done individually. According to the previous definition symbol and a visual background, a gaze situation is expressed as: C_gao=(|<u>AO</u>|&lt;L_a)and(<u>AO</u>@<u>u</u>_<u>a</u>&gt;|<u>AD</u>|*C_a), where A represents the avatar of the participant (for example, the first person's field of view), and O represents the object under consideration. For this gaze situation, the triggering of the defined object behavior is given by the following paradigm logic:

If (S_a=B){

When ((C_gao= True) and (-t&gt;\_g))}{

Initiate the ongoing behavior of O;

If (Δt&gt;10*T_g){

Initiate the behavior of Os long-term gaze;

}

}

When ((C_gao=false) and (Δt&gt;T_g))}{Start the behavior of stopping O;

}

}

In this logic, S_aIs the current state of A, T_gIs the allowable time for gaze, |<u>AO</u>| Is the distance between A and O,<u>AO</u>Is the vector connecting A and O, and -t is C_gaoThe time to maintain this main value. The other symbols are as previously defined.

In this example, the situation (S_a=B) Ensure that only when the participants avatar is in the basic state (for example, not interacting with other avatars), it is eligible for the avatar-object gaze, and the avatar-avatar gaze has priority over the avatar-object gaze. However, it can be confirmed that this situation can be ignored, supplemented by, or replaced by other situations without departing from the principle of the present invention.

Also in this example, the behavior performed by the long-term gaze party is selected for the behavior triggered by ten times the gaze time, which is suitable for the ongoing behavior. It can be confirmed that the multiple can be configured for different objects without departing from the principle of the present invention, and it is not necessarily ten times.

If more than one object meets the gaze condition, attributes and/or rules are preferably used to determine which object gets the participant's attention. If the attribute/rule cannot identify the object that gets attention, then it is better to support other mechanisms. Examples of other mechanisms include providing random selection, enabling the participant to select objects, and supporting multiple object selection as previously described.

2. Priority of non-incarnation objects

As mentioned earlier, this attribute also applies to non-incarnation objects. The attributes of the non-avatar object, like the avatar, are preferably used to (1) transmit data related to the object on a network, (2) process the object, and (3) resolve conflicts between objects for attention. In addition to reducing the processing-oriented load, the use of attributes can increase immersion in the virtual environment. (If increasing immersion is the main goal, then the R2C structure is better used.)

3. Intruded objects

As previously stated, the system is preferably completed to activate the non-avatar object in order to break into the attention of a participant. The interactive control engine associated with FIG. 8 preferably regards this intrusion as attention drift. For example, an application may allow a set of a loud sound and/or a bright light to automatically transfer a participant's gaze vector to the source object. If the intrusion creates a false, surprising gaze situation, and the participant/avatar is in the interactive state, then the participant/avatar will transition to the separated state. If the gaze of the participant/avatar fails to return to the interactive partner within a suitable allowable time, then the participant/avatar will transition from the separated state to the basic state. If the gaze can return to the interactive partner within a suitable allowable time, or the participant issues a notice request or notice consent to the interactive partner, then the participant/avatar will better return to the interactive state. When the participant/avatar is in a state other than basic or interactive, if attention is captured, then the participant/avatar becomes the f-gaze state. If the gaze returns or the participant issues an attention consent or attention request to the interacting partner within a suitable allowable time, then the participant/avatar will return from the f-gaze state to its transition to the f- The state before the gaze state. If the attention drift exceeds the suitable allowable time, then the participant/avatar will transition to the basic state. Just like in the real world, when an object breaks in, the participant decides what they want to do.

If the system allows non-avatar objects to intrude into the attention of a participant/avatar as it is done, then the system preferably also provides a mechanism to resolve conflicts between objects competing for intrusion, for example, among multiple signal sources, each It is possible to cause an intrusion. When similar conflicts occur (for example, when the participant/avatar is not interacting, the conflict between objects), the system can use attributes and/or rules on the object (whether it is an avatar, an object without an avatar, or both). Both) inter-arbitration. Take an example, a signal with the highest intensity to grab the attention. As another example, a participant may choose to allow the audio signal to grab their attention. To give another example, a participant can choose to block a source of disliked signals; however, this blocking is vulnerable to the fact that the system activates the information provider, such as an advertiser, to pay to reduce the restrictions on the source of the provider. As another example, the conflict can be resolved by random selection, and the mechanism can be completed alone or in combination (ie, a mechanism of last resort).

The system can also be implemented to provide priority control of the setting, allowing participants to choose how much they allow non-incarnation objects to automatically grab their attention, and what kind of signal can trigger the grab event. This control can be used to set the lowest threshold that triggers the event signal (for example, a high threshold effectively blocks the signal).

<u>Other sensing</u>

The system described above includes the attention and priority elements, and can be applied to other sensing. This application specifically refers to sensing conditions that have similar visual characteristics, such as hearing and smell. However, it is understandable that this type of sensing can be characterized by different ranges and different sensitivities, for example, sensitivity to the direction of the focus of attention.

Each completed sensing preferably has at least one associated attention space. Each such space is preferably characterized by a sensing ball, sensing cone, sensing vector, focus ball, focus cone and focus vector. Like vision, the radius of a sensing ball is preferably adjusted to reflect a sensing range of the appropriate sensing. The direction of a focus vector represents the center of the attention. The length of the focus vector represents the distance of the focus of attention, and the internal angle of rotation of the focus cone determines the width of the focus of attention. To illustrate, generally speaking, when using the first-person field of view, a participant cannot sense the source signal arranged outside a first-person sensor cone, which may be filled with part or the entire sensor. ball. Furthermore, when compared with the source signal placed outside of any cone, the participant has enhanced the ability to sense the source signal placed inside a focal cone of the participant.

In terms of completion, a system according to the present invention can support attention and priority based on multiple sensing. The calculations for each supported sensing can preferably be implemented independently. As an example, the system can be implemented to assign priority to a sound information source in a sensing space using an auditory signal, while assigning priority to activity and/or images using one or more visual spaces.

The setting attention control can be used, for example, setting the radius for each sensing/focus ball, and setting the length and the angle for each sensing/focus cone. It can also be used to select attention spaces for each media type.

<u>Text-based communication</u>

In a virtual environment that is accustomed to using text-based communication between participants (for example, text-based chat), it is typical for all participants to receive everything near the "speaking" avatar. Post the message. For example, if a first participant publishes a message to a special second participant, then not only the second participant receives the message, but all other participants receive the message. This tends to produce difficulties in the interpretation of a message, because the message is typically outside the context. Sometimes, the message runs too fast to be read comfortably. In this environment, participants typically include tags for the conversation (e.g., the name of the intended recipient and/or the alias of the avatar). (In the following discussion, the participant who publishes a message is sometimes referred to as the "sending participant"; and the participant who is scheduled to receive such a message is sometimes referred to as the "intended recipient".)

In this system, the system preferably extracts the name of the intended recipient (for example, the alias of the intended recipient's avatar) in order to identify the intended recipient of the message. As an example, this extraction can be done in the attention component.

In any case, once the predetermined recipient is identified, the system preferably performs the operation as if the sending participant has sent an attention request control to the predetermined recipient. Then, the intended recipient can choose whether to respond or not. If the predetermined recipient chooses not to respond, the system preferably performs the operation as if the predetermined recipient has sent an inadvertent consent control to the sending participant. If the predetermined recipient chooses to respond, the system preferably performs the operation as if the predetermined recipient has sent an attention consent control to the sending participant. From this point on until the end of the communication, the system handles the communication during the same interaction.

A typical completion shows the follow-up message sent by the sending participant on the screen of the recipient in a selected way so as to distinguish other posted messages. For example, the messages of the interactive dialogue can be distinguished by the following things: (i) punctuation, color (for example, bold), size or font, (ii) by a beep or flash of light indicating guidance, (iii) By the sound of the computer, or (iv) by a combination of the above or other mechanisms. In addition, it is preferable for the participants to choose to make their respective avatars produce another similar object, for example, to initiate an interaction in the visual space.

Another exemplary completion will automatically send the sending participant and the intended recipient to a dedicated chat room.

When the label of the message is changed to another name, note that drift can be detected. The allowable time value can be adjusted according to the need to pay attention to drift in a text-based system. For example, the allowable time can be set to be smaller than the allowable time in other backgrounds. Using a small allowable time, once the tag coup d'etat, the interaction with the original partner ends, and the interaction with the partner displayed on the new tag starts (for example, similar to a participant who changed the tag, it sends an attention request To an intruder).

Participants' names and/or aliases of their avatars can illustrate this situation in a dialogue sentence. A typical completion defines a specific grammar for the label. For example, a tag can be defined as the first word of a message, followed by a colon ":". Other exemplary completions can be used without departing from the principles of the present invention.

<u>Two-dimensional virtual environment</u>

The interaction in the two-dimensional virtual environment is a special case of the interaction in the three-dimensional or above virtual environment. For description, in a two-dimensional virtual environment, the attention and priority element uses a focus cone and a focus vector plan in a two-dimensional space (for example, xy) plane. The conditions and calculations are related to interaction and priority calculations. The previous description here also applies to the two-dimensional space, but the element in the three-dimensional space (for example, the z-axis) is located at 0.

It is understandable that various changes in the details, tools, and arrangements of the parts and steps that have been explained and depicted in order to explain the nature of the present invention can be made by those skilled in the art without departing from the principle and scope of the present invention. Implementation, as expressed in the final scope of the patent application.

<p>8 Virtual Environment System 18 Avatars of Participants</p><p>10 Network system 19 No avatar</p><p>12 Communication channel 20 host</p><p>14 Participant 21 Wiring</p><p>16 Virtual environment 22 Computing device</p><p>17 Other avatars 24 display devices</p><p>26 Input/Output Device 101 Avatar</p><p>28 Screen 102 Visual cone</p><p>29 General Purpose Computer 103 Immersion Points</p><p>30 Processor 104 Vision Ball</p><p>32 Memory system 106 visual vector</p><p>33 Operating system 108 apex</p><p>34 Soft 110 avatar's face</p><p>35 Virtual environment element 112 The body of the avatar</p><p>36 Interactive wiring 114 warp</p><p>37 Other application software 116 Surface</p><p>38 Attention Module 150 Gaze Space</p><p>39 Interactive Control Engine 152 Gaze Cone</p><p>40 Priority Module 154 Gaze Vector</p><p>52 Avatar Attention Element 156 Surface</p><p>54 Priority element 158 Gaze ball</p><p>55 Pay attention to the space 160 surface</p><p>56 Attention sensing space 200 first incarnation</p><p>58 Pay attention to the focal space 201 Virtual environment</p><p>60 Sensor cone 202 second incarnation</p><p>62 Sensor Ball 204 The gaze cone of the first incarnation</p><p>64 Sensing vector 205 The staring vector of the first incarnation</p><p>66 Focus Cone 206 Condensation Cone of the Second Incarnation</p><p>68 Focus ball 207 The gaze vector of the second incarnation</p><p>70 Focus vector 210 vertices</p><p>100 Scan Space 213 Immersion Points</p>

The first figure a shows a block diagram of a virtual environment system, which includes a priority element and an attention element according to the theory of the present invention;

The first figure b shows a block diagram of a virtual environment system, which shows the attention sensing and focus space of the attention element according to the theory of the present invention;

The second picture shows the sensing space;

The third figure shows the sensing space;

The fourth figure shows the focal space contained in a sensing space;

The fifth image shows the focus cones of two avatars, of which only one avatar is placed in the gaze cone of the other avatar;

The sixth image shows the focus cones of two avatars, where each avatar is placed in the gaze cone of the other avatar;

The seventh figure shows the resulting characteristics of the interaction between a focus cone and a plane, which contains the gaze vector of one avatar in the sixth figure and the connection line from the avatar to the other avatar;

The eighth figure shows a state transition diagram combined with an interactive control engine in the first figure a, in which spontaneous and planned interactions are mixed; and

The ninth figure shows a specific embodiment of the priority area of a priority element projected on a plane, the plane including the center of the focus ball and the sensing ball, the focus vector and the sensing vector according to the theory of the present invention.