Dynamically Manipulating An Emoticon or Avatar

CROSS-REFERENCE TO RELATED APPLICATIONS

p-0002This is the first application filed for the present technology.

TECHNICAL FIELD

p-0003The present technology relates generally to communication devices and, more particularly, to messaging applications that enable the use of emoticons or avatars.

BACKGROUND

p-0004Emoticons are small, typically simplistic, images representing a user's facial expression or current mood. For example, a smiley face such as <img id="CUSTOM-CHARACTER-00002" he="2.79mm" wi="2.46mm" file="US20120059787A1-20120308-P00002.TIF" alt="custom-character" img-content="character" img-format="tif" orientation="portrait" inline="no" /> is an emoticon used to signal that the writer of the message is smiling or happy. The emoticon <img id="CUSTOM-CHARACTER-00003" he="2.79mm" wi="2.46mm" file="US20120059787A1-20120308-P00003.TIF" alt="custom-character" img-content="character" img-format="tif" orientation="portrait" inline="no" /> is conventionally used to signal that the writer of the message is unhappy or displeased. These emoticons therefore enable the recipient of e-mail, text messages, etc. to better understand the tenor of the writer's message.

p-0005A similar concept is an avatar which is frequently used on the internet or in gaming to represent a particular user. The avatar is usually in the form of a graphical image, icon or picture which the user selects to represent himself or herself online.

p-0006Conventionally, emoticons and avatars are selected from a predetermined set or palette of prefabricated images, graphics or icons. Because the user is confined to select his or her emoticon (or avatar) from a finite number of possible emoticons (or avatars), the user cannot convey nuances in mood or feeling. A solution to this technical problem would thus be highly desirable.

BRIEF DESCRIPTION OF THE DRAWINGS

p-0007Further features and advantages of the present technology will become apparent from the following detailed description, taken in combination with the appended drawings, in which:

p-0008<figref idrefs="DRAWINGS">FIG. 1</figref> is a schematic depiction of a wireless communications device as one example of a computing device upon which the present technology may be implemented;

p-0009<figref idrefs="DRAWINGS">FIG. 2</figref> is a flowchart depicting main steps of a method of dynamically manipulating an emoticon in accordance with one implementation of the present technology;

p-0010<figref idrefs="DRAWINGS">FIG. 3</figref> is another flowchart depicting main steps of a method of automatically modifying an emoticon in accordance with another implementation of the present technology;

p-0011<figref idrefs="DRAWINGS">FIG. 4</figref> depicts a touch-screen mobile device executing an instant messaging application that enables the user to utilize emoticons and to dynamically manipulate the emoticons;

p-0012<figref idrefs="DRAWINGS">FIG. 5</figref> depicts an enlarged emoticon for dynamic touch-screen manipulation in accordance with one implementation of this technology;

p-0013<figref idrefs="DRAWINGS">FIG. 6</figref> depicts how the emoticon of <figref idrefs="DRAWINGS">FIG. 5</figref> may be manually distorted, stretched or otherwise manipulated using touch input to customize the appearance of the emoticon;

p-0014<figref idrefs="DRAWINGS">FIG. 7</figref> depicts how the emoticon may be customized by the addition of a hat (or other artefact or decorative object) and how that object may itself be manipulated to further customize the appearance of the emoticon; and

p-0015<figref idrefs="DRAWINGS">FIG. 8</figref> depicts the automatic generation or modification of an emoticon by parsing the text of an instant message to search for words from which the user's mood may be inferred.

p-0016It will be noted that throughout the appended drawings, like features are identified by like reference numerals.

DETAILED DESCRIPTION

p-0017In general, the present technology provides a method, computer-readable medium and device for dynamically manipulating or modifying graphic user representations, such as emoticons or avatars used during an electronic mail conversation, instant message chat or other such dialogue. The modification or manipulation of these graphic user representations enables the user to convey nuances of mood and feeling rather than be confined to select an appropriate emoticon or avatar from a palette of predetermined emoticons or avatars.

p-0018Thus, one aspect of the present technology is a method of dynamically modifying a graphic user representation such as an emoticon or avatar. The method entails displaying the graphic user representation, receiving user input, and modifying a visual appearance of the graphic user representation to indicate a current user mood in response to the user input.

p-0019Another aspect of the present technology is a computer-readable medium comprising instructions in code which when loaded into memory and executed on a processor of a mobile device is adapted to perform the steps of the foregoing method.

p-0020Yet another aspect of the present technology is a computing device that has a processor coupled to memory for executing a communication application that enables composition and transmission of messages. The computing device includes a display for displaying a graphic user representation, such as an emoticon or avatar, and a user input device for receiving user input to dynamically manipulate a visual appearance of the graphic user representation to thereby create a modified graphic user representation that indicates a current user mood, wherein the display presents the modified emoticon or avatar.

p-0021The details and particulars of these aspects of the technology will now be described below, by way of example, with reference to the attached drawings.

p-0022<figref idrefs="DRAWINGS">FIG. 1</figref> is a high-level depiction of a generic mobile device or wireless communications device <b>100</b> as one example of a computing device upon which the present technology may be implemented. As shown by way of example in <figref idrefs="DRAWINGS">FIG. 1</figref>, the novel mobile device, which is designated generally by reference numeral <b>100</b>, includes a processor (or microprocessor) <b>110</b> for executing one or more applications, memory in the form of flash memory <b>120</b> and/or random access memory (RAM) <b>130</b> for storing the one or more applications and related data, and a user interface <b>140</b> with which the user interacts with the application(s). The user interface <b>140</b> includes a display screen <b>150</b> (e.g. an LCD screen or a touch-sensitive display screen). The device <b>100</b> may include a keypad or keyboard <b>155</b> and a thumbwheel, trackball or trackpad <b>160</b>.

p-0023Optionally, where the mobile device is a wireless communications device, the device <b>100</b> would further include a radiofrequency (RF) transceiver chip <b>170</b> and antenna <b>172</b>. Optionally, where the device is a voice-enabled wireless communications device, such as, for example, a smartphone or cell phone, the device would further include a microphone <b>180</b> and a speaker <b>182</b>.

p-0024This device <b>100</b> may optionally include a GPS receiver chipset <b>190</b> or other location-determining subsystem.

p-0025For greater certainty, and for the purposes of this specification, the expression “mobile device” is meant to encompass a broad range of portable, handheld or mobile electronic devices such as smart phones, cell phones, satellite phones, PDA's or Pocket PCs, computing tablets, laptops, etc. that have communication capabilities for exchanging textual messages.

p-0026For the purposes of this specification, the expression “communication device” is meant to encompass any computing device, mobile device or electronic device that enables a user to compose, send, receive, and read textual messages such as e-mail, MMS, SMS, etc.

p-0027It bears emphasizing that while this technology may be implemented on a wireless communications device or mobile device, it may also be implemented on any wired computing device, such as a standard wired desktop computer or personal computer.

p-0028In accordance with various implementations of the present technology, the processor <b>110</b> and memory <b>120</b>, <b>130</b> of this mobile device <b>100</b> may be configured to dynamically modify or manipulate a graphic user representation, such as an emoticon or avatar. The dynamic modification of the emoticon or avatar may be done manually (in response to user input on the device), or automatically (in response to keywords parsed from the text of the message that has been sent or is being composed by the user), or as a combination of both. A broad overview of the dynamic modification of emoticons or avatars will now be presented with reference to the flowcharts of <figref idrefs="DRAWINGS">FIG. 2</figref> and <figref idrefs="DRAWINGS">FIG. 3</figref>.

p-0029<figref idrefs="DRAWINGS">FIG. 2</figref> is a flowchart depicting main steps of a method of dynamically manipulating an emoticon in accordance with one implementation of the present technology. In general terms, <figref idrefs="DRAWINGS">FIG. 2</figref> shows that one process of dynamically modifying an emoticon may be triggered or initiated by user input, e.g. a command from the user that causes the device to incorporate an emoticon into a text message, e-mail, etc. The command may be, for example, a keystroke or a series of keystrokes that cause a predefined emoticon to be displayed. For example, the series of keystrokes may entail typing a colon, followed by a dash, followed by a closing parenthesis which the computer recognizes as an instruction to provide a smiley face emoticon <img id="CUSTOM-CHARACTER-00004" he="3.13mm" wi="2.46mm" file="US20120059787A1-20120308-P00001.TIF" alt="custom-character" img-content="character" img-format="tif" orientation="portrait" inline="no" />. Alternatively, the command may cause a palette of predefined emoticons to be displayed (from which the user selects the emoticon that he or she desires). Incorporation of the emoticon according to any of these aforementioned techniques is performed at step <b>200</b> in the flowchart of <figref idrefs="DRAWINGS">FIG. 2</figref>. Subsequently, at step <b>210</b>, the user may signal to the device that the user wishes to dynamically manipulate or modify the emoticon that has been previously selected and displayed. This may entail receiving user input at the device in the form of a command that signals to the device that the user wishes to customize the emoticon. Subsequently, and in response to step <b>210</b>, the device provides at step <b>220</b> a manipulation interface that may comprise a plurality of onscreen manipulation widgets, tools, menus, buttons, or other such means for manipulating, distorting, stretching, coloring, or decorating the emoticon. The various ways in which the emoticon may be manipulated are presented below.

p-0030<figref idrefs="DRAWINGS">FIG. 3</figref> is another flowchart depicting main steps of a method of automatically modifying an emoticon in accordance with another implementation of the present technology. Specifically, this figure depicts a process whereby an emoticon is automatically generated or modified by the device in response to inferences being drawn about the user's mood.

p-0031Inferences may be drawn about the user's mood, condition, state of mind, etc. by parsing the text of messages that the user has sent and received. For example, an artificial intelligence module operating on the device may detect certain words, phrases or expressions in the messages. Upon detecting a phrase such as, for example, “I'm having a bad day”, the device may infer that the prevailing emotion or mood of the user is one of displeasure or exasperation. The device may then choose to automatically insert a suitable emoticon such as, for example, a sad face emoticon <img id="CUSTOM-CHARACTER-00005" he="2.79mm" wi="2.46mm" file="US20120059787A1-20120308-P00003.TIF" alt="custom-character" img-content="character" img-format="tif" orientation="portrait" inline="no" />. Similarly, upon detecting a reply along the lines of “Sorry to hear that”, the AI module of the computing device may automatically provide an emoticon that suggests empathy. If an avatar is already being employed for the user, this avatar may be dynamically modified as the conversation progresses. Thus, as the mood of the discussion changes, as discerned or inferred by the AI module, the avatar may be dynamically modified accordingly. The degree to which these changes are modulated by previous moods may be configurable.

p-0032The mood may also be gleaned from presence information that is published by the user via a presence service. Thus, the presence information for a user may provide indicia of mood that can be imported into the text message exchange for the purposes of modifying the avatar and/or for the purposes of dynamically generating or modifying an emoticon. Conversely, mood detection of the user from his dialogue may be exported to the presence service. Thus, the present technology can interact intimately with a presence service.

p-0033Other external factors may be used to deduce or infer the user's mood or current state of mind. For example, mood may be inferred from the user's current location and/or his current activity as declared in a calendar entry, social media site, blog, personal web page or other source. Mood may also be inferred from other factors like the prevailing local weather, news or RSS feeds that may be relevant to the user or to user groups or news groups to which he subscribes. For example, if the AI module of the device detects that the current location is a vacation destination, the current weather there is sunny and warm, and the user's calendar shows that the user is indeed on vacation, then the AI module may infer from these circumstances that the user is likely to be in a good mood. If on the other hand, for example, the user's favourite sports team has lost, the weather is rainy, and the user is scheduled to be a meeting, then the AI module may infer that the mood of the user is likely to be poor.

p-0034From these textual or circumstantial inferences, the device may automatically modify the avatar or emoticon without direct manipulation by the user. This is represented by the flowchart of <figref idrefs="DRAWINGS">FIG. 3</figref>. As was the case with the flowchart of <figref idrefs="DRAWINGS">FIG. 2</figref>, the process may begin (at step <b>230</b>) when the user instructs the device to incorporate an emoticon into the message. As noted above, this may involve a command or series of keystrokes to invoke a particular emoticon or it may involve selecting one emoticon from a palette of predefined emoticons. Subsequently, at step <b>240</b>, the language of the messages sent, received or actively being composed is parsed to infer emotions or mood. At step <b>250</b>, the device automatically modifies the emoticon (or avatar) based on the inference as to emotion or mood.

p-0035Automatic modification of emoticons may be activated, deactivated and reactivated by configuring the device settings in an options page. Automatic modification of emoticons may be configured to operate on only the emoticon that has been last selected by the user, on the most recent set of emoticons selected by the user, or on the entire of palette of emoticons. Again, the extent of this automatic modification may be reconfigurable.

p-0036In a variant of the two main methods presented in <figref idrefs="DRAWINGS">FIG. 2</figref> and <figref idrefs="DRAWINGS">FIG. 3</figref>, the device may automatically modify the emoticon and request validation of this automatic modification by the user. The user may, for example, touch the screen to validate or annul the proposed modification.

p-0037To summarize, the device may firstly be configured to generate a new emoticon or avatar. Generation of a new emoticon or avatar may be manual or automatic, i.e. it may be based on user input, or based on textual or circumstantial inferences. Secondly, the device may be configured to dynamically modify an existing emoticon or avatar. Modification may be either manual or automatic. Manual modification involves user input that triggers the display of onscreen manipulation tools which the user then employs to manipulate, distort or otherwise customize the emoticon or avatar. Automatic modification of an existing emoticon or avatar entails dynamically varying the appearance of the existing emoticon or avatar based on textual or circumstantial inferences drawn by the AI module of the device.

p-0038The subsequent figures in this application depict various specific implementations of this novel technology.

p-0039<figref idrefs="DRAWINGS">FIG. 4</figref> depicts a touch-screen mobile device executing an instant messaging (IM) application. The display shows an IM conversation between Amy and Bob. Each user has chosen to incorporate emoticons after their respective messages. As shown by way of example in the particular implementation presented in <figref idrefs="DRAWINGS">FIG. 4</figref>, the emoticons are simple smiley faces, although it should be appreciated that visually more intricate avatars may be employed. The user interface of the device presented by way of example in this figure shows that the device enables the user to either (i) select an emoticon from a palette by clicking a user interface element on the touch screen or (ii) dynamically manipulate the selected emoticon by clicking a different user interface element at the bottom of the touch screen. Where the dynamic modification user interface element is touched, the device displays the emoticon and a tool set for onscreen manipulation. Any other dedicated key, button or user interface element may be provided for this purpose (i.e. triggering the dynamic modification tool set).

p-0040<figref idrefs="DRAWINGS">FIG. 5</figref> depicts an enlarged emoticon for dynamic touch-screen manipulation in accordance with one implementation of this technology. As depicted by way of example in this figure, the touch screen device displays the avatar or emoticon in an enlarged view. The user may distort, stretch, squeeze, rotate or color the emoticon to thereby customize it to the specific mood of the user. This enables the user to convey nuanced moods and feelings rather than being confined to predefined emoticons. The user interface may also provide options to add a feature to the emoticon (e.g. add hair, add a hat, add a moustache, etc.) or to changes other visual characteristics like color, size, etc. Still further options (not shown) may be provide to vary the specific facial features (e.g. the user may vary the appearance of the emoticon's eyes, ears, nose, mouth, etc.)

p-0041<figref idrefs="DRAWINGS">FIG. 6</figref> depicts how the emoticon of <figref idrefs="DRAWINGS">FIG. 5</figref> may be manually distorted, stretched or otherwise manipulated using touch input to customize the appearance of the emoticon. Onscreen manipulation tools (e.g. arrows to show the selected direction of stretching or squeezing, curved arrows to show the direction of rotation, or other such symbols) may be displayed to facilitate the user's task of distorting, stretching, squeezing or otherwise manipulating the emoticon. Various buttons or keys may be displayed for various functions, e.g. save, undo, etc.

p-0042<figref idrefs="DRAWINGS">FIG. 7</figref> depicts how the emoticon may be customized by the addition of visual features such as the addition of hair or clothing such as, for example, a hat. The visual feature may be selected from a palette of predefined artefacts or objects such as, for example, a flag, sword, sports equipment, food or other decorative object. This figure also shows how that object (in this case, the hat) may itself be manipulated to further customize the appearance of the emoticon.

p-0043<figref idrefs="DRAWINGS">FIG. 8</figref> depicts the automatic generation or modification of an emoticon by parsing the text of an instant message to search for words from which the user's mood may be inferred. In this example, an IM conversation between Bob and Amy is parsed by the AI module of the device. The first message contains the word “hope” which signifies a positive outlook. A smiley face emoticon is automatically incorporated for this message. The final message contains the phrase “I'm so angry” which the AI module recognizes as implying anger. Accordingly, the emoticon generated is one showing a face of displeasure.

p-0044In another implementation, instead of inserting emoticons after each message, a single avatar can be displayed representing the user's mood. This single avatar, which may be displayed anywhere onscreen, is dynamically modified either manually (by user input) or automatically (based on textual or circumstantial inferences).

p-0045The foregoing examples show how emoticons and avatars can be manipulated on a touch screen device. Specific touch screen gestures may be programmed so as to be recognizable by the AI module of the device to mean that a certain emoticon distortion or manipulation is desired.

p-0046The foregoing method steps may be implemented as software, i.e. as coded instructions stored on a computer readable medium which performs the foregoing steps when the computer readable medium is loaded into memory and executed by the microprocessor of the mobile device. A computer readable medium can be any means that contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus or device. The computer-readable medium may be electronic, magnetic, optical, electromagnetic, infrared or any semiconductor system or device. For example, computer executable code to perform the methods disclosed herein may be tangibly recorded on a computer-readable medium including, but not limited to, a floppy-disk, a CD-ROM, a DVD, RAM, ROM, EPROM, Flash Memory or any suitable memory card, etc. The method may also be implemented in hardware. A hardware implementation might employ discrete logic circuits having logic gates for implementing logic functions on data signals, an application-specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

p-0047It should be appreciated that this technology may be implemented on any other (non-touch-sensitive mobile device, communication device or computing device). In one implementation, a user input device may be provided with dedicated keys (hot keys) for manipulating the emoticon or avatar. The dedicated keys may be haptic, force-sensitive or force-feedback keys that distort the emoticon or avatar in proportion to the force applied to the key(s). For example, the device may have force-sensitive keys on opposite sides of the device. The user squeezes the keys to visually squeeze (distort) the emoticon.

p-0048This new technology has been described in terms of specific implementations and configurations which are intended to be exemplary only. Persons of ordinary skill in the art will, having read this disclosure, readily appreciate that many obvious variations, refinements and modifications may be made without departing from the inventive concept(s) disclosed herein. The scope of the exclusive right sought by the Applicant(s) is therefore intended to be limited solely by the appended claims.