System and method of providing voice-message call service.
Abstract
Provided are a system and method of providing a voice-message call service. A mobile device that performs a call with an external mobile device comprises a control unit configured to obtain text, the text converted from voice data that is exchanged between the mobile device and the external mobile device, during the call between the mobile device and the external mobile device, and obtain input text input to the mobile device and provided text that is received from the external mobile device; and a display unit configured to arrange the text, the input text, and the provided text and display the arranged text, input text, and provided text on a screen of the device, during the call between the mobile device and the external mobile device.
Term
8.7 yearsleft in the term
Expires 22 May 2035.
- Priority
- Filed
- Granted
- Today
- Expires
15 claims: 3 independent, 12 dependent
- 1CLAIMS REIVINDICACIONES Habiéndose descrito la invención como antecede, se reclama como propiedad lo contenido en las siguientes reivindicaciones:Having described the invention as above, the content of the following claims is claimed as property: 1. A mobile device that carries out a call with an external mobile device, characterized in that it comprises: 1. Un dispositivo móvil que lleva a cabo una llamada con un dispositivo móvil externo, caracterizado porque comprende: a control unit configured to get text, the text converted from voice data that is exchanged between the mobile device and the external mobile device, during the call between the mobile device and the external mobile device, and get input text entered to the mobile device and provided text that is received from the external mobile device;and a display unit configured to lay out the text, input text and provided text and to display the laid out text, input text, and provided text on a screen of the device, during the call between the mobile device and the external mobile device. una unidad de control configurada para obtener texto, el texto convertido a partir de datos de voz que se intercambian entre el dispositivo móvil y el dispositivo móvil externo, durante la llamada entre el dispositivo móvil y el dispositivo móvil externo, y obtener texto de entrada ingresado al dispositivo móvil y texto proporcionado que se recibe desde el dispositivo móvil externo;y una unidad de presentación visual configurada para disponer el texto, el texto de entrada y el texto proporcionado y presentar visualmente el texto dispuesto, el texto de entrada y el texto proporcionado en una pantalla del dispositivo, durante la llamada entre el dispositivo móvil y el dispositivo móvil externo.
- 12A method of a mobile device that carries out a call with an external mobile device, characterized in that it comprises:12. Un método de un dispositivo móvil que lleva a cabo una llamada con un dispositivo móvil externo, caracterizado porque comprende: obtaining text, the text converted from voice data that is exchanged between the mobile device and the external mobile device, during the call between the mobile device and the external mobile device;obtener texto, el texto convertido a partir de datos de voz que se intercambian entre el dispositivo móvil y el dispositivo móvil externo, durante la llamada entre el dispositivo móvil y el dispositivo móvil externo;get input text that is entered on the device;obtener texto de entrada que se introduce en el dispositivo;get provided text that is received from the external mobile device;and arrange the text, input text, and text obtener texto proporcionado que se recibe desde el dispositivo móvil externo;y disponer el texto, el texto de entrada y el texto 147 provided and display the arranged text, input text and provided text on a screen of the device, during the call between the mobile device and the external mobile device. 147 proporcionado y presentar visualmente el texto dispuesto, el texto de entrada y el texto proporcionado en una pantalla del dispositivo, durante la llamada entre el dispositivo móvil y el dispositivo móvil externo.
- 14A mobile device characterized in that it comprises:14. Un dispositivo móvil caracterizado porque comprende: a transceiver;and a processor configured to control the transceiver to establish a voice message communication session with an external mobile device, the voice message communication session supporting the transmission and reception of voice data and text data between the mobile device and the external mobile device. un transceptor;y un procesador configurado para controlar el transceptor para establecer una sesión de comunicación de mensajes de voz con un dispositivo móvil externo, la sesión de comunicación de mensajes de voz soportando la transmisión y recepción de datos de voz y datos de texto entre el dispositivo móvil y el dispositivo móvil externo.
Independent claims3
527 paragraphs in 1 section, as filed
(54) Title: SYSTEM AND METHOD TO PROVIDE VOICE MESSAGING CALL SERVICE.
(54) Title: SYSTEM AND METHOD OF PROVIDING VOICE-MESSAGE CALL SERVICE.
(57) Summary
A system and method are provided for providing a voice message calling service. A mobile device that carries out a call with an external mobile device comprises a control unit configured to obtain text, the text converted from voice data that is exchanged between the mobile device and the external mobile device, during the call between the mobile device and the external mobile device, and get input text entered to the mobile device and provided text that is received from the external mobile device; and a display unit configured to lay out the text, input text and provided text and to display the laid out text, input text, and provided text on a screen of the device, during the call between the mobile device and the external mobile device.
(57) Abstract
Provided are a system and method of providing a voice-message cali Service. A mobile device that performs a cali with an external mobile device comprises a control unit configured to obtain text, the text converted from voice data that is exchanged between the mobile device and the external mobile device, during the cali between the mobile device and the external mobile device, and obtain input text input to the mobile device and provided text that is received from the external mobile device; and a display unit configured to arrange the text, the input text, and the provided text and display the arranged text, input text, and provided text on a screen of the device, during the cali between the mobile device and the external mobile device.
SYSTEM. AND METHOD OF PROVIDING VOICE MESSAGE CALL SERVICE
Field of Invention
Aspects of the exemplary embodiments relate to a system and method for providing a voice message call service and, more particularly, to a system and method for providing a voice call and a text service over a voice message service. voicemail call.
Background of the Invention
Due to developments in multimedia technology and network technology, one user can communicate with another user using various devices. In particular, the user can have a conversation with another user using a voice call service and a text messaging service.
However, an environment in which the user can have a conversation with the other user using a device, such as a noisy environment or a car environment, may be unsuitable for a voice call or a text exchange, respectively.
Consequently, there is a demand for a technology that allows the user to jointly use a voice call service and a voice exchange service.
Ref. 271984 text, or switch and efficiently use the voice call service and the text exchange service.
Brief Description of the Invention Solution to the problem
Aspects of the exemplary embodiments include a system and a method for providing a voice message call service, whereby a voice call and a text service may be co-provided through the voice message call service.
Aspects of the exemplary embodiments include a system and method for providing a voice message call service capable of distinguishing between texts indicating a conversation between users during a voice message call.
Aspects of the exemplary modes include a system and method for providing a voice message call service capable of advising a user of a change in a call mode during a voice message call.
Other aspects will be set forth in part in the description that follows and, in part, will be apparent from the description, or may be learned by practicing the presented modalities.
Brief Description of Figures
The above and other aspects will become apparent and will be more easily appreciated from the following description of the exemplary modalities, taken in conjunction with the attached figures, in which:
Figure 1 illustrates a voice message call service according to an exemplary embodiment.
Figure 2 is a flow chart of a method for displaying text that is converted from a voice and text that is entered by a user, the method being carried out by a first device during a message call from voice, according to an exemplary modality.
Figures 3A to 5 illustrate examples in which a voice message call function is activated, according to exemplary embodiments.
FIG. 4 illustrates an example in which a voice message call function of the first device is activated when the first device receives a call, according to an exemplary embodiment.
Figure 5 illustrates an example in which a voice message call function of the first device is activated during a voice call, according to an exemplary embodiment.
Figure 6 is a flowchart of a method for carrying out a voice message call with a second device whose voice message call function is not activated, the method being carried out by the first device whose function voice message call is activated, in accordance with an exemplary embodiment.
FIG. 7 is a flow chart of a method for carrying out a voice message call in a mutual manner, the method being carried out by the first device and the second device, according to an exemplary embodiment.
Fig. 8 is a flow chart of a method for carrying out a voice message call in a mutual manner, the method being carried out by the first device and the second device, according to another exemplary embodiment.
FIG. 9 is a flow chart of a method for carrying out a voice message call in a mutual manner, the method being carried out by the first device and the second device, according to another exemplary embodiment.
FIG. 10 is a flowchart for displaying a chat window during a voice message call, the method being carried out by the first device, according to an exemplary embodiment.
Figures 11A and 11B illustrate examples of a chat window displayed during a voice message call, in accordance with exemplary embodiments.
Figure 12 illustrates an example in which the first device and the second device exchange a text that is converted from a voice, through a specific chat application, according to an example modality.
Figure 13 is a flowchart of a method for determining a chat application that runs interoperating with a voice call when a voice message call function is activated, the method being carried out by the first device, of according to an exemplary modality.
Figure 14 is a flowchart of a method for determining a chat application to be used during a voice message call, the method being carried out by the first device, in accordance with an example mode.
Figure 15 is a flow chart of a chat application installation method to be used during a voice message call, the method being carried out by the first device, in accordance with an exemplary embodiment.
Figure 16A illustrates an example in which the first device activates a voice message call function while the first device uses a chat service, in accordance with an exemplary embodiment.
FIG. 16B illustrates an example in which the first device initiates a voice message call, according to user input through a chat application user list screen, according to an exemplary embodiment.
Figure 17 is a flow chart of a method for conducting a voice message call while the first device and the second device use a talk service, the method being performed by the first device and the second device. , according to an exemplary modality.
Figures 18 to 21 illustrate examples in which a call mode is changed during a voice message call, in accordance with exemplary embodiments.
Fig. 22 is a flow diagram of a method of switching a mode of a voice message call, the method being carried out by the first device, in accordance with an exemplary embodiment.
Fig. 2-3 is a flow chart of a method for conducting a voice message call during a voice mode, the method being performed by the first device, in accordance with an exemplary embodiment.
FIG. 24 is a flow chart of a method for carrying out a voice message call during a text mode, the method being carried out by the first device, in accordance with an exemplary embodiment.
Figure 25 is a flow chart of a method of recognizing a calling mode of the second device, the method being carried out by the first device, according to an exemplary embodiment.
Figure 26 is a flowchart of a method for conducting a voice message call, the method being performed by the first device operating in a voice mode and the second device operating in a text mode. , according to an exemplary modality.
Figure 27 is a flow chart of a method for conducting a voice message call, the method being performed by the first device operating in a voice mode and the second device operating in a text mode. , according to an exemplary modality.
Figure 28 is a flow chart of a method for conducting a voice message call, the method being performed by the first device operating in a voice mode and the second device operating in a text mode. , according to an exemplary modality.
Figures 29A, 29B and 29C illustrate examples of a combination mode which is one of the calling modes of a voice message call, according to empirical modalities.
Fig. 30 illustrates an example in which a call mode of the first device that carries out a voice message call is changed from a voice mode to a combination mode or is changed from a combination mode to a voice mode. according to an exemplary modality.
Fig. 31 illustrates an example in which a calling mode of the first device carrying out a voice message call is changed from a text mode to a combination mode or is changed from a combination mode to a text mode, according to an exemplary modality.
FIG. 32 is a flow chart of a method of switching a mode of a voice message call, the method being carried out by the first device, according to an exemplary embodiment.
FIG. 33 is a flow chart of a method for carrying out a voice message call during a combining mode, the method being carried out by the first device, in accordance with an exemplary embodiment.
Figure 34 is a flow chart of a method for conducting a voice message call with the second device, the method being performed by the first device operating in a combination mode, according to an exemplary mode .
Figure 3-5A is a flow chart of a method of advising a first user of a change in a calling mode of the second device and recommending a first user to change a calling mode, the method being carried out by the first device, according to an exemplary embodiment.
Figure 35B is a flowchart of a method of advising a first user of a change in the input / output functions of the second device during a combination mode, and recommending to the first user to change a calling mode, the method being carried out by the first device, according to an example modality.
Figure 36 illustrates an example in which the first device recommends a first user to change a calling mode from a voice mode to a text mode, in accordance with an exemplary mode.
Fig. 37 illustrates an example in which the first device recommends a first user to change a calling mode from a text mode to a voice mode, in accordance with an exemplary mode.
Fig. 3-8 is a flow chart of a method of generating a call list in relation to voice message calls, the method being carried out by the first device, in accordance with an exemplary embodiment.
FIG. 39 is a flow diagram of a method for displaying text indicating a conversation between a first user and a second user through a first device call list, in accordance with an exemplary embodiment.
Figures 40 to 42 illustrate examples of a call list, according to exemplary embodiments.
FIG. 43 illustrates an example in which the first device and the second device carry out a voice message call through the server, according to an exemplary embodiment.
Figure 44 is a flowchart of a method of supporting a voice message call between the first device and the second device by converting the voices of the first and second users to texts, the method being carried out by the server, according to an exemplary modality.
Figure 4-5 is a flow chart of a method of supporting a voice message call between the first device and the second device by converting the texts of the first and second users into voices, the method being carried out by the server , according to an exemplary modality.
Figure 46 is a flow chart of a method of storing voice data and text data related to a voice message call between the first device and the second device, the method being carried out by the server, according with a modality eg emplar.
Fig. 47 illustrates in the example that the first device, the second device and a third device carry out a voice message call with each other, according to an exemplary embodiment.
Fig. 48 illustrates an example in which the first device, the second device and the third device carry out a voice message call with each other through the server, according to an exemplary embodiment.
Figures 49 and 50 are block diagrams of the first device, according to exemplary embodiments.
Figure 51 is a block diagram of the server, according to an exemplary embodiment.
Brief Description of the Invention
According to one aspect of an exemplary embodiment, a mobile device is provided that makes a call with an external mobile device, the mobile device including a control unit configured to obtain converted voice data that is exchanged between the mobile device and the external mobile device and converted into converted text, during the call between the mobile device and the external mobile device, and obtaining input text input to the mobile device and provided text that is received from the external mobile device; and a display unit for arranging the converted text, input text and provided text and displaying the converted text, input text and provided text on a screen of the device, during the call between the mobile device and the external mobile device .
According to one aspect of an exemplary embodiment, there is provided a method of a mobile device making a call with an external mobile device, the method including obtaining converted voice data that is exchanged between the mobile device and the device. external mobile and become converted text, during the call between the mobile device and the external mobile device, get the input text that is entered into the device, get the provided text provided from the external mobile device and organize the converted text, input text and provided text and visually present the converted text, input text and provided text on a device screen, during the call between the mobile device and external mobile device.
In accordance with one aspect of an exemplary embodiment, a system is provided that provides a call between a first mobile device and a second mobile device, the system including a server that includes a communication unit configured to receive a voice from a first user from the first mobile device; and a control unit configured to convert the voice of the first user to text, wherein the communication unit is further configured to transmit the converted text to the second device; the first device mobile being configured to provide the voice of the first user to the server; and the second mobile device configured to display the converted text, which is transmitted to the second mobile device, in a chat window that is displayed on a screen of the second mobile device during the call, wherein the chat window shows a conversation between the first user and a second user.
Hereinafter, exemplary embodiments will be described in more detail with reference to the accompanying figures. However, exemplary embodiments can be incorporated in many different ways and should not be construed as limited to the exemplary embodiments set forth herein; rather, these exemplary embodiments are provided so that this description will be complete and detailed, and will fully convey the concept of the exemplary embodiments to those skilled in the art. In the following description, well-known functions or constructions will not be described in detail so as not to obscure the exemplary embodiments with unnecessary detail. Furthermore, throughout the description, the same reference numbers in the figures indicate similar elements.
Throughout the description, it will also be understood that when one element is referred to as connected to another element, it may be directly connected to the other element or electrically connected to the other element, while intermediate elements may also be present.
Throughout the description, a voice message call service may indicate a service that co-provides a voice call service and a chat (messaging) service. A user can make a voice call with a user of another device using one device and can also exchange a message with the user of the other device during the voice call. For example, a calling mode of the voice message call service may include at least two selected from a voice mode, a text mode, and a combination mode.
Throughout the description, the voice mode may indicate a calling mode during which a user can make a voice call with another user.
Throughout the description, the text mode may indicate a calling mode during which a user can communicate with another user through an exchange of a text message.
Throughout the description, the combination mode may indicate a calling mode during which a user can make a voice call with another user and. You can simultaneously exchange a text message with the other user using a single call mode call connection.
As an example, during merge mode, the user can make a voice call with the other user using a speakerphone feature of one device and can simultaneously communicate with the other user by exchanging a text message with the other user at through the same communication connection. As another example, during the merge mode, the user can carry out a voice call with the other user without using the speakerphone function and can simultaneously communicate with the other user by exchanging a text message with the other user about the same. communication connection.
Furthermore, various techniques can be used to transmit voice data and text data between devices. For example, voice data or text data can be transmitted and received between the devices using a mobile communication service of which the user is a member or wireless techniques (for example, Wi-Fi or Bluetooth) that can be used in the devices. For example, when using LTE or Wi-Fi communication technology, voice data and text data can be transmitted and received between devices over a packet network. In this case, voice data can be transmitted using VoLTE technology. Alternatively, when WCDMA is used, voice data can be transmitted over a circuit network and text data can be transmitted over a packet network. Voice data and text data can be transmitted through separate communication connections. When using the packet network, a communication connection between the devices can be a TCP / IP connection. When using the circuit network, the communication connection between the device can be a call connection.
As used herein, the term and / or includes any combination of one or more of the associated related elements. Expressions such as at least one of, by preceding a list of items, modify the entire list of items and do not modify individual items in the list.
Hereinafter, exemplary embodiments will be described in more detail with reference to the accompanying figures.
Figure 1 illustrates a voice message call service according to an exemplary embodiment.
Referring to Figure 1, the first device 1000 can carry out a voice message call with a second device 2000. The voice message call can be initiated as a single connection or communication session that supports both voice service like the courier. The first device 1000 and the second device 2000 can carry out a voice call with each other and during the voice call, the first device 1000 can display text indicating a conversation between users on a screen of the first device 1000. Furthermore During the call, the second device 2000 may also display the text indicating the conversation between the users on a screen of the second device 2000. For example, the first device 1000 and the second device 2000 can display chat windows showing the conversation between the users on the screens of the first device 1000 and the second device 2000, respectively. The chat window can be a window through which messages are transmitted and received. The first user of the first device 1000 and the second user of the second device 2000 can communicate with each other by entering messages in the chat window.
Accordingly, the user of the first device 1000 and the user of the second device 2000 can co-use a voice call service and a chat service during the voice message call. In this regard, continuous switching between voice and messaging can be carried out during the call of voice messages. Alternatively, both voice and messaging can take place simultaneously during the voice message call.
The voice message call can comprise one or more channels, over which voice data and text data can be transmitted. For example, voice data can be transmitted over a voice channel and text data can be transmitted through a voice channel. Alternatively, a data channel can also be included in the voice message call, for data transmission.
Each of the first device 1000 and the second device 2000 can be, but are not limited to, a smartphone, a tablet personal computer (PC), a PC, a smart television (TV), a mobile phone, a digital assistant personal computer (PDA), a laptop, a media player, a micro-server, a global positioning device (GPS), an electronic book terminal, a terminal for digital broadcasting, a navigation device, a kiosk, MP3 player, camera, portable device, and other mobile or non-mobile computing devices. In addition, each of the first device 1000 and the second device 2000 can include various devices such as an electronic whiteboard, a touch table, etc. that can receive a touch input. In addition, each of the first device 1000 and the second device 2000 may be a watch, glasses, a hair band or a ring having a communication function and a data processing function.
Figure 2 is a flow chart of a method of displaying a text that is converted from a voice and a text that is entered by a user, the method being carried out by the first device 1000 during a call from voice message, according to an exemplary modality.
In step S200, the first device 1000 may activate a voice message call function. When the first device 1000 sends a call to the second device 2000, the first device 1000 can activate the voice message call function. Alternatively, when the first device 1000 receives a call from the second device 2000, the first device 1000 can activate the voice message call function. Alternatively, while the first device 1000 is making a call with the second device 2000, the first device 1000 can activate the voice message call function.
Also, when the voice message call function is activated, the first device 1000 may advise the second device 2000 that the voice message call function is activated on the first device 1000. In this case, the first device 1000 can advising the second device 2000 of a calling mode of the voice message call of the first device 1000. The calling mode of the voice message call will be described later.
In addition, when the voice message call function is activated, various techniques can be used to transmit voice data and text data. For example, voice data or text data can be transmitted and received between the first device 1000 and the second device 2000 using a mobile communication service of which the user is a member or wireless techniques (for example, Wi-Fi or Bluetooth. ) that can be used on the devices. For example, when using an LTE or Wi-Fi communication technology, voice data and text data can be transmitted and received between the first devices 1000 and the second device 2000 over a packet network. In this case, voice data can be transmitted using VoLTE technology. Alternatively, when WCDMA is used, voice data can be transmitted over a circuit network and text data can be transmitted over a packet network.
Voice data and text data can be transmitted through separate communication connections. When using the packet network, a communication connection between the devices can be a TCP / IP connection. When using the circuit network, the communication connection between the device can be a call connection.
In step S210, the first device 1000 can obtain texts that are converted from a voice of a user of the first device 1000 and a voice of a user of the second device 2000. The first device 1000 can convert the voice of the user of the first device. 1000 device into a text and thus you can get the converted text. In addition, the first device 1000 can receive the voice of the user of the second device 2000 from the second device 2000, can convert the voice into a text, and thus can get the converted text. However, a method for obtaining the converted text, the method being carried out by the first device 1000, is not limited thereto. The voice of the user of the first device 1000 or the voice of the user of the second device 2000 can be converted into a text by various devices or a server that mediates communication between the first device 1000 and the second device 2000 and the first device 1000 can receive the converted text from the various devices and the server.
In step S220, the first device 1000 can obtain a text that is entered by a user. The first device 1000 may obtain a text that is entered by the user of the first device 1000 during the voice message call. Furthermore, during the voice message call, the first device 1000 may receive, from the second device 2000, a text that is entered into the second device 2000 by the user of the second device 2000.
In step S230, the first device 1000 can arrange the converted texts and input texts in a sequential time order and can display the converted texts and input texts on a screen of the first device 1000 during a call between users. The first device 1000 may arrange the converted texts and the input texts in temporal order, based on the input times of the voices and texts. For example, based on a time that the user of the first device 1000 enters the voice into the first device 1000, a time that the user of the second device 2000 enters the voice into the second device 2000, a time that the user of the first device 1000 enters the text in the first device 1000 and a time when the user of the second device
2000 input the text into the second device 2000, the first device 1000 can arrange the converted texts and input texts.
Figures 3A to 5 illustrate examples in which a voice message call function is activated, according to exemplary embodiments.
FIG. 3A illustrates an example in which a voice message call function of the first device 1000 is activated when the first device 1000 sends a call, in accordance with an exemplary embodiment.
Referring to FIG. 3A, a user of the first device 1000 can enter a phone number of the second device 2000 and can then select a voice message call button 30. Accordingly, the first device 1000 can Activate the voice message call function and you can initiate a call to the second device 2000.
Figure 3B illustrates an example in which a voice message call function of the first device 1000 is activated based on user input to select a button in a phone book that is displayed on a screen of the first device 1000, according to an exemplary modality.
Referring to Figure 3B, the first device 1000 may display the phone book on the screen of the first device 1000 and a user of the first device 1000 may select a voice message call button 32 included in a field of a list of users in which a phone number of the second device 2000 is displayed, wherein the list of users is included in the displayed phone book. Accordingly, the first device 1000 can activate the voice message call function and can initiate a call to the second device 2000.
FIG. 4 illustrates an example in which a voice message call function of the first device 1000 is activated when the first device 1000 receives a call, in accordance with an exemplary embodiment.
Referring to Figure 4, when the first device 1000 receives a call from the second device 2000, a user of the first device 1000 can select a voice message call button 40. Accordingly, the first device 1000 can activate the function. voice message call and you can receive the call from the second device 2000. Alternatively, when the first device 1000 initiates a voice message call, the second device 2000 can automatically accept the voice message call.
Figure 5 illustrates an example where a voice message call function of the first device
1000 it is activated during a voice call, according to an exemplary embodiment.
Referring to Figure 5, a user of the first device 1000 can select a voice message call button 50 during a voice call with a user of the second device 2000. Accordingly, the first device 1000 can activate the call function. voice message and you can make a voice message call with the second device 2000. Alternatively, the second device 2000 may activate the voice message call function during a voice call with the first device 1000. In this regard, the call may be initiated as a voice call and the voice call may be initiated during the call. voice call.
Figure 6 is a flowchart of a method for conducting a voice message call with the second device 2000 whose voice message call function is not activated, the method being carried out by the first device 1000 of whose voice message call function is activated, according to an example mode.
Referring to Fig. 6, the first device 1000 whose voice message call function is activated can carry out the voice message call with the second device 2000 and the second device.
2000 whose voice message call function is not activated can make a voice call with the first device 1000.
In step S600, the first device 1000 is connected by call with the second device 2000. The first device 1000 can transmit a call connection request to the second device 2000 or can receive a call connection request from the second device 2000, so that the first device 1000 can be call connected with the second device 2000.
In step S610, the first device 1000 can activate a voice message call function. The first device 1000 may activate the voice message call function, based on user input, or automatically. In step S610, the first device 1000 activates the voice message calling function after the first device 1000 is connected by call with the second device 2000. However, one or more exemplary modalities are not limited to that. For example, the first device 1000 may activate the voice message call function when the first device 1000 sends a call or receives a call or while the first device 1000 is making a call.
In step S620, the first device 1000 can transmit a voice from a user of the first device 1000 (hereinafter, the user of the first device 1000 is referred to as the first user of the first device 1000) to the second device 2000. The first device 1000 it can obtain the voice of the first user through a microphone and can transmit the obtained voice to the second device 2000.
In step S630, the first device 1000 can convert the voice of the first user to text. The first device 1000 can convert the first user's speech to text using various speech-to-text (STT) techniques.
In step S640, the first device 1000 can receive a voice from a user of the second device 2000 (hereinafter, the user of the second device 2000 is referred to as a second user of the second device 2000) from the second device 2000. In this step , the second device 2000 can obtain the voice of the second user using a microphone in the second device 2000 and can transmit the voice obtained from the second user to the first device 1000.
In step S650, the first device 1000 can convert the second user's voice to text. The first device 1000 can convert the second user's voice into text using the various STT techniques.
In step S660, the first device 1000 can arrange the converted texts in a temporal order. For example, the first device 1000 may arrange the text converted from the voice of the first user and the text converted from the voice of the second user, according to a time when the voice of the first user is input to the first device. 1000 and a time when the voice of the second user is received. Also, for example, the first device 1000 can arrange the text converted from the voice of the first user and the text converted from the voice of the second user, according to the time when the voice of the first user is input. at the first device 1000 and a time when the second user's voice is input to the second device 2000. However, one or more exemplary embodiments are not limited thereto.
In step S670, the first device 1000 can display a chat window that includes the texts arranged on a screen of the first device 1000. The first device 1000 can display the chat window on the screen of the first device 1000 while the first device 1000 perform the voice message call. Furthermore, the first device 1000 may display the chat window on the screen of the first device 1000 or it may hide the chat window on the screen of the first device 1000, depending on the calling mode of the voice message call.
FIG. 7 is a flow chart of a method for carrying out a voice message call in a mutual manner, the method being carried out by the first device 1000 and the second device 2000, according to an exemplary embodiment. In the embodiment of Figure 7, the first device 1000 can convert a voice of a first user and a voice of a second user into texts - and the second device 2000 can convert the voice of the first user and the voice of the second user into texts.
In step S700, the first device 1000 and the second device 2000 are connected to each other. The first device 1000 may be call connected to the second device 2000 by transmitting a call connection request to the second device 2000 or by receiving a call connection requirement from the second device 2000.
In step S705, the first device 1000 may activate a voice message calling function. The first device 1000 may activate the voice message call function, based on user input. In step S705, the first device 1000 activates the voice message calling function after the first device 1000 is connected by call with the second device 2000. However, one or more exemplary embodiments are not limited thereto. For example, the first device 1000 may activate the voice message calling function when the first device 1000 sends a call or receives a call or while the first device 1000 is making a call.
In step S710, the second device 2000 can activate a voice message call function. The second device 2000 may activate the voice message call function, based on user input. In step S710, the second device 2000 activates the voice message calling function after the second device 2000 is connected by call with the first device 1000. However, one or more exemplary embodiments are not limited thereto. For example, the second device 2000 may activate the voice message call function when the second device 2000 sends a call or receives a call or while the second device 2000 carries out a call.
In step S715, the first device 1000 can transmit the voice of the first user to the second device 2000. The first device 1000 can obtain the voice of the first user through a microphone in the first device 1000 and can transmit the obtained voice to the second device 2000.
In step S720, the first device 1000 can convert the voice of the first user to text. The first device 1000 can convert the voice of the first user to text using various STT techniques.
In step S725, the second device 2000 can convert the voice of the first user, which is received from the first device 1000, into a text. The second device 2000 can convert the voice of the first user to text using the various STT techniques.
In step S730, the second device 2000 can transmit the voice of the second user to the first device 1000. The second device 2000 can obtain the voice of the second user through a microphone in the second device 2000 and can transmit the obtained voice to the first device. device 1000.
In step S735, the first device 1000 can convert the second user's voice, which is received from the second device 2000, into a text. The first device 1000 can convert the second user's voice to text using various STT techniques.
In step S740, the second device 2000 can convert the second user's voice into text. The second device 2000 can convert the second user's voice to text using various STT techniques.
In the S745 operation, the first device 1000 can arrange the converted texts in a temporal order. For example, the first device 1000 may arrange the text converted from the voice of the first user and the text converted from the voice of the second user, according to a time when the voice of the first user is input to the first device. 1000 and a time when the voice of the second user is received. Furthermore, for example, the first device 1000 may sequentially arrange the text converted from the voice of the first user and the text converted from the voice of the second user, according to the moment when the voice of the first user is input. in the first device 1000 and a time when the second user's voice is input to the second device 2000. However, one or more exemplary embodiments are not limited thereto.
In step S750, the second device 2000 can arrange the converted texts in a temporal order. For example, the second device 2000 may sequentially order the text converted from the voice of the first user and the text converted from the voice of the second user, according to a time when the voice of the second user is input to the second device 2000. and a time when the voice of the first user is received. Also, for example, the second device 2000 may arrange the text converted from the voice of the first user and the text converted from the voice of the second user, according to the time when the voice of the second user is entered into the second device 2000 and a time when the voice of the first user is input to the first device 1000. However, one or more exemplary embodiments are not limited thereto.
In step S755, the first device 1000 can display a chat window including the texts arranged on a screen of the first device 1000. The first device 1000 can display the chat window on the screen of the first device 1000 while the first device 1000 make the voice call. Furthermore, the first device 1000 may display the chat window on the screen of the first device 1000 or it may hide the chat window on the screen of the first device 1000, depending on the calling mode of the voice message call.
In step S760, the second device 2000 can display a chat window that includes the texts arranged on a screen of the second device 2000. The second device 2000 can display the chat window on the screen of the second device 2000 while the second device 2000 make the voice call. Furthermore, the second device 2000 may display the chat window on the second device 2000 screen or may hide the chat window on the second device 2000 screen, depending on the calling mode of the voice message call.
FIG. 8 is a flow chart of a method for carrying out a voice message call in a mutual manner, the method being carried out by the first device 1000 and the second device 2000, according to another exemplary embodiment. . In the embodiment of Figure 8, the first device 1000 can convert a voice of a first user to text and the second device 2000 can convert a voice of a second user to text.
In step S800, the first device 1000 and the second device 2000 are connected to each other.
In operation S805, the first device 1000 can activate a voice message call function, and in operation S810, the second device 2000 can activate a voice message call function. In the embodiment of Figure 8, the first device 1000 and the second device 2000 activate the voice message call functions, respectively, after the first device 1000 and the second device 2000 are connected by call to each other, but a or more exemplary modes are not limited to that, as the devices 1000 and 2000 can automatically initiate the voice message calling functions.
In step S815, the first device 1000 can transmit the voice of the first user to the second device 2000. The first device 1000 can obtain the voice of the first user through a microphone in the first device 1000 and can transmit the obtained voice to the second device 2000.
In step S820, the first device 1000 can convert the voice of the first user to text. The first device 1000 can convert the voice of the first user to text using various STT techniques.
In step S825, the first device 1000 can transmit the text of the first user to the second device 2000.
In step S830, the second device 2000 can transmit the voice of the second user to the first device 1000. The second device 2000 can obtain the voice of the second user through a microphone in the second device 2000 and can transmit the obtained voice to the first device 1000.
In step S835, the second device 2000 can convert the second user's voice into text. The second device 2000 can convert the second user's voice to text using various STT techniques.
In step S840, the second device 2000 can transmit the text of the second user to the first device 1000.
In step S845, the first device 1000 can arrange the converted text and the received text in a temporal order. For example, the first device 1000 may arrange the text that is converted by the first device 1000 from the voice of the first user and the text of the second user that is received from the second device 2000, in temporal order.
In step S850, the first device 1000 can display a chat window including the texts arranged on a screen of the first device 1000. The first device 1000 can display the chat window on the screen of the first device 1000 while the first device 1000 make the voice call.
In step S855, the second device 2000 may arrange the converted text and the received text in a temporal order. For example, the second device 2000 may arrange the text that is converted from the second user's voice by the second device 2000 and the first user's text that is received from the first device 1000, in time order.
In step S860, the second device 2000 can display a chat window that includes the texts arranged on a screen of the second device 2000. The second device 2000 can display the chat window on the screen of the second device 2000 while the second device 2000 make the voice call.
FIG. 9 is a flow chart of a method for carrying out a voice message call in a mutual manner, the method being carried out by the first device 1000 and the second device 2000, according to another exemplary embodiment. In the embodiment of Figure 9, the first device 1000 can convert a voice of a first user and a voice of a second user into texts.
Steps S900 to S930 of Fig. 9 correspond to steps S800 to S830 of Fig. 8, therefore a detailed description thereof is omitted here.
In step S935, the first device 1000 can convert the second user's voice to text. The first device 1000 can convert the second user's voice to text using various STT techniques.
In step S940, the first device 1000 can transmit the second user's text to the second device 2000.
In step S945, the first device 1000 can arrange the converted texts in a temporal order. For example, the first device 1000 may arrange the text that is converted by the first device 1000 from the voice of the first user and the text that is converted by the first device 1000 from the voice of the second user, in time order.
In step S950, the first device 1000 can display a chat window including the texts arranged on a screen of the first device 1000. The first device 1000 can display the chat window on the screen of the first device 1000 while the first device 1000 make the voice call.
In step S955, the second device 2000 can arrange the converted texts in a temporal order. The second device 2000 may arrange the text of the first user that is received from the first device 1000 and the text of the second user, in temporal order.
In step S960, the second device 2000 can display a chat window that includes the texts arranged on a screen of the second device 2000. The second device 2000 can display the chat window on the screen of the second device 2000 while the second device 2000 make the voice call.
FIG. 10 is a flowchart of visual presentation of a chat window during a voice message call, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
In operation S1000, the first device 1000 can activate a voice message call function. When the first device 1000 initiates a call to the second device 2000, the first device 1000 can activate the voice message call function. Alternatively, when the first device 1000 receives a call from the second device 2000, the first device 1000 can activate the voice message call function. Alternatively, while the first device 1000 carries out a call with the second device 2000, the first device 1000 can activate the voice message call function.
In step S1010, the first device 1000 can obtain texts that are converted from a voice of a first user and a voice of a second user. The first device 1000 can convert a voice of a user of the first device 1000 into a text and thus can get the converted text. Furthermore, the first device 1000 can receive a voice of a user of the second device 2000 from the second device 2000, can convert the received voice into a text, and thus can get the converted text. However, a method for obtaining a converted text, the method being carried out by the first device 1000, is not limited thereto. The voice of the user of the first device 1000 and the voice of the user of the second device 2000 can be converted into the texts by various devices or a server and the first device 1000 can receive the converted texts from the various devices and the server. For example, texts can be received from device 2000 or the server, which can mediate a communication session between the first device 1000 and the second device 2000.
In step S1020, the first device 1000 can back up the original voice data of the converted texts. The first device 1000 may back up first user's voice data corresponding to the text that is converted from the first user's voice. In addition, the first device 1000 may back up second user's voice data corresponding to the text that is converted from the second user's voice.
In step S1030, the first device 1000 may display a chat window on a screen of the first device 1000 during a call between the first user and the second user. The first device 1000 may display the chat window on the screen of the first device 1000 to display the content of a conversation between the first user and the second user.
In step S1040, the first device 1000 can obtain a text that is entered by the first user through the chat window. In addition, the first device 1000 may receive, from the second device 2000, a text that the second user enters to the second device 2000 during the voice message call.
In step S1050, the first device 1000 can arrange the converted texts and input texts in a temporal order and can display them in the conversation window. The first device 1000 may arrange the converted texts and input texts in temporal order, based on the input times of the voices and texts. For example, the first device 1000 may arrange the converted texts and the input texts, based on a time when the first user enters the voice in the first device 1000, a time when the second user enters the voice in the second device. 2000, when the first user enters the text on the first device 1000 and a time when the second user enters the text on the second device 2000.
In step S1060, the first device 1000 can distinguish between the converted texts and the input texts and can display them. The first device 1000 can distinguish the converted text from the first user's voice and the converted text from the second user's voice from the text entered by the first user and the text entered by the second user and can display them. In this regard, a user can distinguish between text input by a user and voice input by a user that are converted into text form and displayed.
In order to distinctively display the converted texts, the first device 1000 may display a separate icon around the converted texts. Alternatively, the first device 1000 may distinctively display a color, thickness, or font of the converted texts. However, one or more exemplary modalities are not limited to that.
In step S1070, when at least one converted text is selected from the converted texts, the first device 1000 can output original voice data of the selected converted text. When the first user 1000 selects the converted text in the conversation window, the first device 1000 can extract the original voice data of the selected converted text from a memory and can output the original extracted voice data through a loudspeaker in the first device 1000.
In step S1080, the first device 1000 may back up a text among the arranged texts, wherein the text is obtained for a preset period of time before a predetermined event occurs. For example, the default event may include an end of a call between the first device 1000 and the second device 2000, receipt of user input for a text backup, or the like. As an example, when the call between the first device 1000 and the second device 2000 ends, the first device 1000 can back up a text that is obtained by the first device 1000 for five minutes before the call ends. As another example, when the first device 1000 receives user input for the text backup, the first device 1000 can back up text that is obtained by the first device 1000 for three minutes before user input is received. . However, an event type and a time value that are set for text backup are not limited to this.
The first device 1000 can back up all the texts laid out on a server 3000. In this case, the first device 1000 can back up some of the texts laid out on the first device 1000 and can back up all the texts laid out on the server 3000. Alternatively, the first device 1000 can back up some of the laid out texts and all the laid out texts on the server 3000.
Referring to FIG. 10, the first device 1000 backs up the text between the arranged texts, wherein the text is obtained during the preset period of time before the predetermined event occurs, but one or more exemplary modes are not limited. to that. The first device 1000 can analyze a conversation between users by performing various natural language-based analysis methods on all arranged texts and can extract a portion of arranged texts indicating important contents of the conversation between users. In addition, the first device 1000 can back up a text indicating the extracted part of the conversation in the first device 1000.
Figures 11A and 11B illustrate examples of a chat window displayed during a voice message call, in accordance with exemplary embodiments.
Fig. 11A indicates an example in which a text that is converted from a voice is distinctively displayed in the chat window displayed during the voice message call, according to an exemplary embodiment.
Referring to FIG. 11A, the chat window can be displayed on a screen of the first device 1000 during the voice message call, and Sure and What's Up? which are texts converted from a user's voice can be displayed in the chat window. Also, to indicate that what are you going to have? is the text that is converted from the voice, the first device 1000 can display an icon 110 around What are you going to have ?.
When a first user touches What are you going to have ?, the first device 1000 can emit original voice data corresponding to the text touched What are you going to have? through a loudspeaker of the first device 1000. The first device 1000 can emit the voice of the user who pronounces What are you going to have ?. In this regard, the original voice of the user may be buffered by the first device 1000, the second device 2000, or a server that mediates a connection between the devices. Accordingly, the buffered speech can be reproduced on demand.
Accordingly, although there is a typographical error in the text that is converted from the user's voice, the user can listen to the original voice data of the converted text and thus correctly recognize the content of the conversation. user.
Furthermore, the first device 1000 can indicate a level of conversion reliability in relation to the text that is converted from the text. For example, when Claro's reliability is greater than the reliability of ¿Qué vas a haber ?, a size and thickness of Claro letters can be greater than a size and thickness of ¿Qué vas a haber ?. However, one or more exemplary modalities are not limited to them, and therefore, a color and a font of Claro letters can be displayed distinctively. Additionally, a specific object indicating a reliability value can be displayed around the converted text.
Fig. 11B indicates an example in which a text that is converted from a voice is distinctively displayed by a unit of a word in the chat window displayed during the voice message call, according to a exemplary mode.
Referring to FIG. 11B, the first device 1000 can distinctively display the reliability of What are you going to have ?, which is a text converted from a voice, by a unit of a word. For example, when the reliability of Qué y vas is higher than the reliability of ay there in the text What are you going to have ?, a size and thickness of the letters Qué y vas may be visually presented larger than a size and a thickness of letters ay have. However, one or more exemplary modalities are not limited to them, and therefore, a color and a font of the letters Qué y vas can be displayed distinctively.
In addition, when a first user touches What of What are you going to have ?, the first device 1000 can output original voice data corresponding to a touched word What? through a loudspeaker of the first device 1000. The first device 1000 can emit a voice of a user saying the word What. Consequently, a user can selectively request voice corresponding to a portion of a text message to be played.
Figure 12 illustrates an example in which the first device 1000 and the second device 2000 exchange text that is converted from a voice, through a specific chat application, according to an exemplary embodiment.
Referring to FIG. 12, a chat application 120 may run on the first device 1000 and a chat application 122 may run on the second device 2000. The chat application 120 and the chat application 122 may be the same applications or they can be compatible with each other.
In addition, the first device 1000 can enter a text that is converted from a voice and is obtained by the first device 1000 in the chat application 120 and can transmit the input text to the chat application 122 of the second device 2000 to through the chat app 120.
In addition, the second device 2000 can enter a text that is converted from a voice and is obtained by the second device 2000 in the chat application 122 and can transmit the input text to the chat application 120 of the first device 1000 to via chat app 122.
Figure 13 is a flowchart of a method for determining a chat application that will run interoperating with a voice call when a voice message call function is activated, the method being carried out by the first device 1000, according to an exemplary modality.
In step S1300, the first device 1000 can activate the voice message calling function. When the first device 1000 sends a request to initiate the call to the second device 2000, the first device 1000 can activate the voice message call function. Alternatively, when the first device 1000 receives a call from the second device 2000, the first device 1000 can activate the voice message call function. Alternatively, while the first device 1000 carries out a call with the second device 2000, the first device 1000 can activate the voice message call function.
In step S1310, the first device 1000 can determine a chat application. The first device 1000 can determine the chat application to be used during a voice message call. As an example, the first device 1000 may determine a chat application that is executable by interoperating with the voice message call function of the first device 1000, such as the chat application to be used during the voice message call. As another example, the first device 1000 may determine a chat application that is installed on all of the first device 1000 and the second device 2000, as the chat application to be used during the voice message call. However, one or more exemplary modalities are not limited to that. Alternatively, a user can select a singing application to be executed from one or more chat applications when a call is initiated or a call request is received, as described below.
In step S1320, the first device 1000 can execute the determined chat application, and in step S1325, the first device 1000 can input a text, which is converted from a voice, into the executed chat application. For example, the first device 1000 can install a plug-in for automatic text input in the running chat application and can enter the converted text from speech into the chat application through the installed plug-in. However, one or more exemplary modalities are not limited to that.
In step S1330, the first device 1000 may display a chat window on a screen of the first device 1000 during a call between users. The first device 1000 can arrange, in the conversation window, a text that is converted from a voice of the first user, a text that is entered in the first device 1000 by the first user, a text that is converted from a voice of a second user and a text that is entered to the second device 2000 by the second user.
Furthermore, the first device 1000 may display the chat window on the screen of the first device 1000 or it may hide the chat window on the screen of the first device 1000, depending on the calling mode of the voice message call.
FIG. 14 is a flow chart of a method for determining a chat application to be used during a voice message call, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
In operation S1400, the first device 1000 can request from the second device 2000 a list of chat applications installed on the second device 2000 and in operation S1410 the second device 2000 can provide the list of chat applications installed on the second device. 2000 to the first 1000 device.
In step S1420, the first device 1000 can select a chat application among the chat applications that are installed in the first device 1000, wherein the chat application is compatible with a chat application among the chat applications in the list. received from the second device 2000.
For example, a chat application compatible with chat application 'A' may be the same chat application as chat application Ά '. Also, as another example, the chat app that is compatible with the chat app Ά 'may be a different chat app from the chat app.
Ά ', but able to exchange a message with the chat app Ά'.
Referring to FIG. 14, the first device 1000 requests the second device 2000 for the list of chat applications and thus receives the list of chat applications from the second device 2000, but one or more exemplary modes are not available. limited to them. When the first device 1000 and the second device 2000 are connected for a voice message call, the first device 1000 can receive the list of chat applications from the second device 2000 without a separate request.
Figure 15 is a flow diagram of a chat application installation method to be used during a voice message call, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
In operation S1500, the first device 1000 may request from the second device 2000 a list of chat applications installed on the second device 2000, and in operation S1510, the second device 2000 may provide the list of chat applications installed on the device. second device 2000 to first device 1000.
In step S1520, the first device 1000 can install a chat application that is compatible with a chat application included in the list of chat applications received from the second device 2000. The first device 1000 can select the chat application that is compatible with the chat application included in the list of the chat applications received from the second device 2000 and can download the selected chat application from a predetermined server or the second device 2000. Also, the first device 1000 can install the downloaded chat application on the first device 1000.
Referring to FIG. 15, the first device 1000 requests the second device 2000 for the list of chat applications and thus receives the list of chat applications from the second device 2000, but one or more exemplary modalities do not. they are limited to them. When the first device 1000 and the second device 2000 are connected for a voice message call, the first device 1000 can receive the list of chat applications from the second device 2000 without a separate request.
Fig. 16 illustrates an example in which the first device 1000 activates a voice message call function while the first device 1000 uses a chat service, according to an exemplary embodiment.
Referring to the. Figure 16A, the first device 1000 can run a chat application and can use a predetermined chat service. In addition, when a first user selects a button 160 displayed on a chat application launch screen, the first device 1000 may activate the voice message call function.
FIG. 16B illustrates an example in which the first device 1000 initiates a voice message call in accordance with user input through a chat application user list screen, in accordance with an exemplary embodiment.
Referring to FIG. 16B, the first device 1000 can run the chat application and can display the list of users of the chat application. In addition, the first device 1000 may receive a user input to select a button 165 for a voice call with John in the user list. Consequently, the first device 1000 can initiate a voice message call with a Juan device.
Figure 17 is a flow chart of a method of conducting a voice message call while the first device 1000 and the second device 2000 use a chat service, the method being performed by the first device 1000 and the second device 2000, according to an exemplary embodiment.
In step S1700, the first device 1000 and the second device 2000 are connected to each other for communication for a chat service. In addition, in step S1710, the first device 1000 may display a chat window on a screen of the first device 1000 and in step S1720, the second device 2000 may display a chat window on a screen of the second device 2000. Then the first device 1000 and the second device 2000 can exchange a message.
In operation S1730, the first device 1000 can activate a voice message call function, and in operation S1740, the second device 2000 can activate a voice message call function.
Also, in step S1750, the first device 1000 and the second device 2000 can carry out a voice message call with each other.
Figures 18 to 21 illustrate examples in which a call mode is changed during a voice message call, in accordance with exemplary embodiments.
The calling mode of the voice message call can include a voice mode and a text mode. The voice mode can indicate a mode during which a user can make a voice call with another user and the text mode can indicate a mode during which the user can communicate with the other user by exchanging a message. of text.
Figure 18 illustrates an example in which the first device 1000 automatically changes the calling mode from voice mode to text mode during the voice message call, in accordance with an exemplary mode.
Referring to FIG. 18, when the first device 1000 is adjacent to a user's face, the first device 1000 can turn off a screen of the first device 1000 and can operate in the voice mode. Then, when the first device 1000 is away from the user's face, the first device 1000 can change the calling mode from voice mode to text mode and can display a chat window on the screen of the first device 1000. The first device 1000 can automatically detect the orientation and position of the first device, and switch between voice mode and text mode as required.
Fig. 19 illustrates an example in which the first device 1000 automatically switches the calling mode from the text mode to the voice mode during the voice message call, in accordance with an exemplary mode.
Referring to FIG. 19, when the first device 1000 is away from a user's face, the first device 1000 can display a chat window on a screen of the first device 1000 and can operate in text mode. Then, when the first device 1000 approaches the user's face, the first device 1000 can change the call mode from text mode to voice mode and can turn off the screen of the first device 1000. Again, the first device 1000 can automatically detect the orientation and position of the first device, and switch between voice mode and text mode as required.
Figure 20 illustrates an example in which the first device 1000 manually changes the calling mode from voice mode to text mode during the voice message call, in accordance with an exemplary mode.
Referring to Figure 20, while the first device 1000 operates in the voice mode, when a first user selects a button 200 displayed on a screen of the first device 1000, the first device 1000 can change the calling mode from the voice mode to text mode and can display a chat window on the screen of the first device 1000.
Figure 21 illustrates an example in which the first device 1000 manually changes the calling mode from the text mode to the voice mode during the voice message call, according to an exemplary mode.
Referring to FIG. 21, while the first device 1000 operates in text mode, when a first user selects a button 210 displayed on a screen of the first device 1000, the first device 1000 can change the calling mode from the mode. text to speech mode and you can turn off the screen of the first 1000 device.
Figure 22 is a flow diagram of a method for changing a mode of a voice message call, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
In step S2200, the first device 1000 may determine whether a calling mode of the voice message call is a voice mode. As an example, the first device 1000 can determine if the first device 1000 is adjacent to a face of a first user using a proximity sensor on the first device 1000 and therefore can determine if the call mode is the mode voice. As another example, the first device 1000 may determine whether the first user selects a button to switch to voice mode, and thus may determine whether the calling mode is voice mode.
As a result of the determination in step S2200, when the first device 1000 determines that the calling mode is the voice mode, in step S2210, the first device 1000 can activate a speaker and a microphone. Consequently, the first device 1000 can obtain a voice of the first user through the activated microphone, and can emit a voice of a second user through the activated speaker.
In step S2220, the first device 1000 hides a chat window on a screen of the first device 1000. For example, the first device 1000 can hide the chat window on the screen of the first device 1000 by deactivating the screen of the first device 1000. Even if the display is off, the first device 1000 can still buffer text equivalents of the voice conversation, in case the voice mode changes to text mode. Text equivalents can be buffered on the first device 1000, second device 2000, or on a server.
As a result of the determination in step S2200, if the calling mode is not the voice mode, in step S2230, the first device 1000 can determine whether the calling mode of the voice message call is a text mode. . As an example, the first device 1000 can determine if the first device 1000 is away from the face of the first user using the proximity sensor on the first device 1000 and therefore can determine if the call mode is the call mode. text. As another example, the first device 1000 can determine whether the first user selects a button to switch to text mode, and therefore can determine whether the call mode is text mode.
As a result of the determination in step S2230, when the first device 1000 determines that the calling mode is the text mode, in step S2240, the first device 1000 can activate the speaker and microphone.
In step S2250, the first device 1000 can display the chat window on the screen of the first device 1000. Furthermore, the first device 1000 can display, in the chat window, a text that is entered by the first user at through the chat window. Furthermore, the first device 1000 may display, in the conversation window, a text that is converted from the second user's voice and a text that is entered to the second device 2000 by the second user.
In step S2260, the first device 1000 can determine whether the calling mode is changed. In step S2260, if the first device 1000 determines that the calling mode is changed, the first device 1000 may carry out step S2200.
FIG. 23 is a flow chart of a method for conducting a voice message call during a voice mode, the method being performed by the first device 1000, in accordance with an exemplary embodiment.
In step S2300, the first device 1000 can determine whether the first device 1000 is adjacent to a face of a first user. The first device 1000 can determine if the first device 1000 is adjacent to the face of the first user using a proximity sensor included in the first device 1000.
In step S2310, the first device 1000 can activate a speaker and a microphone. The first device 1000 can determine that a calling mode of the voice message call is a voice mode and therefore can activate the speaker and microphone on the first device 1000. Accordingly, the first device 1000 can receive input from a voice of the first user through the microphone, and can output a voice from a second user through the speaker.
In step S2320, the first device 1000 can convert the voice that is input from the first user into a text. The first device 1000 can convert the voice of the first user to text using various STT techniques.
In step S2330, the first device 1000 hides a chat window on a screen of the first device 1000. The first device 1000 can hide the chat window on the screen of the first device 1000 by deactivating the screen of the first device 1000.
In step S2340, the first device 100 can recognize a call mode of the second device 2000. The second device 2000 can transmit information indicating the call mode of the second device 2000 to the first device 1000 and the first device 1000 can acknowledge the call. of the second device 2000, based on the information indicating the call mode that is received from the second device 2000.
In step S2350, the first device 1000 can selectively transmit, to the second device 2000, at least one of the voice that is entered by the first user and the text that is converted from the voice input by the first user. As an example, when the calling mode of the second device 2000 is a voice mode, the first device 1000 may transmit the voice that is entered by the first user to the second device 20 00. As another example, when the calling mode of the second device 2000 is a text mode, the first device 1000 may transmit, to the second device 2000, the text that is converted from the voice input by the first user. However, one or more exemplary modalities are not limited to that.
FIG. 24 is a flow chart of a method for carrying out a voice message call during a text mode, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
In step S2400, the first device 1000 can determine whether the first device 1000 is away from a face of a first user. The first device 1000 can determine if the first device 1000 is away from the face of the first user using a proximity sensor included in the first device 1000.
In step S2410, the first device 1000 can inactivate a speaker and a microphone. The first device 1000 may determine that a calling mode of the voice message call is a text mode, and thus may disable the speaker and microphone.
In step S2420, the first device 1000 may display a chat window on a screen of the first device 1000. The first device 1000 may display, on the screen of the first device 1000, the chat window showing a conversation between the first user and second user.
In step S2430, the first device 1000 can convert a text that is entered by the first user into a voice. The first device 1000 can convert the text of the first user to speech using various text-to-speech (TTS) techniques. In this case, the first device 1000 can determine a pitch of the converted voice of the first user, in consideration of a gender, an age, etc. of the first user.
In step S2440, the first device 1000 can recognize a call mode of the second device 2000. The second device 2000 can transmit information indicating the call mode of the second device 2000 to the first device 1000 and the first device 1000 can acknowledge the call. of the second device 2000, based on the information indicating the call mode that is received from the second device 2000.
In step S2450, the first device 1000 can selectively transmit, to the second device 2000, at least one of the text that is entered by the first user and the voice that is converted from the text input by the first user. As an example, when the calling mode of the second device 2000 is a voice mode, the first device 1000 can transmit the voice that is converted from the text input by the first user to the second device 2000. For another example, when the call of the second device 2000 is a text mode, the first device 1000 may transmit the text that is entered by the first user to the second device 2000. However, one or more exemplary modes are not limited thereto. .
Fig. 2-5 is a flow chart of a method of recognizing a calling mode of the second device 2000, the method being carried out by the first device 1000, according to an example mode.
In step S2500, the first device 1000 may request the second device 2000 for the call mode of the second device 2000. The first device 1000 may periodically request the second device 2000 for the call mode of the second device 2000, but one or more Non-exemplary modalities are limited thereto. Alternatively, the first device 1000 may automatically request the second device 2000 for call mode when initiating a call.
Furthermore, in operation S2510, the second device 2000 can recognize its calling mode and in operation S2520, the second device 2000 can transmit information about the calling mode of the second device.
2000 to the first 1000 device.
Referring to FIG. 25, the second device 2000 transmits the information about the calling mode of the second device 2000, in response to the request of the first device 1000, but one or more exemplary modes are not limited thereto. For example, although the second device 2000 does not receive the request from the first device 1000, when the call mode of the second device 2000 is changed, the second device 2000 can transmit information about the changed call mode from the second device 2000 to the first device 1000. .
In step S2530, the first device 1000 can recognize the calling mode of the second device 2000. The first device 1000 can recognize whether the calling mode of the second device 2000 is a voice or text mode, based on information about the call from the second device 2000 that is received from the second device 2000.
In step S2540, the first device 1000 can selectively transmit a voice from a first user or a text from the first user to the second device 2000. As an example, when the call mode of the second device 2000 is voice mode, the first Device 1000 can transmit the voice of the first user to the second device 2000. As another example, when the calling mode of the second device 2000 is text mode, the first device 1000 may transmit the text of the first user to the second device 2000.
FIG. 26 is a flow chart of a method of conducting a voice message call, the method being performed by the first device 1000 operating in a voice mode and the second device 2000 operating in a voice mode. text mode, according to an exemplary mode. In the embodiment of FIG. 26, the first device 1000 can convert a voice of a first user to a text and the second device 2000 can convert a text of a second user to a voice.
In step S2600, the first device 1000 and the second device 2000 are connected to each other. The first device 1000 may transmit a call connection request to the second device 2000 or it can receive a call for call connection from the second device 2000, so that the first device 1000 can be connected by call with the second device 2000.
In step S2605, the first device 1000 can activate a voice message call function. The first device 1000 may activate the voice message call function, based on user input. In step S2605, the first device 1000 activates the voice message calling function after the first device 1000 is call connected with the second device 2000. However, one or more exemplary embodiments are not limited thereto. For example, the first device 1000 may activate the voice message call function when the first device 1000 sends a call or receives a call, or while the first device 1000 carries out a call.
In step S2610, the second device 2000 may activate a voice message calling function. The second device 2000 may activate the voice message call function, based on user input. In step S2610, the second device 2000 activates the voice message calling function after the second device 2000 is connected by call with the first device 1000. However, one or more exemplary embodiments are not limited thereto. For example, the second device 2000 may activate the voice message call function when the second device 2000 sends a call or receives a call or while the second device 2000 carries out a call.
In operation S2615, the first device 1000 can operate in the voice mode. When the first device 1000 approaches a face of the first user or receives user input to allow the first device 1000 to operate in the voice mode, the first device 1000 can operate in the voice mode.
In step S2620, the first device 1000 can activate a microphone and speaker in the first device 1000 and can hide a chat window to display a conversation between the first user and the second user on a screen of the first device 1000.
In step S2625, the second device 2000 can operate in the text mode. When the second device 2000 is away from a face of the second user or receives user input that allows the second device 2000 to operate in text mode, the second device 2000 can operate in text mode.
In step S2630, the second device 2000 can deactivate a speaker and a microphone in the second device 2000 and can display a chat window to display a conversation between the first user and the second user on a screen of the second device 2000.
In step S2635, the first device 1000 can receive input from the first user's voice, and in step S2640, the first device 1000 can convert the first user's voice into text.
In step S2645, the first device 1000 can transmit, to the second device 2000, the text that is converted from the voice of the first user. The first device 1000 can recognize that the second device 2000 is operating in the text mode and can transmit the text that is converted from the voice of the first user to the second device 2000.
In step S2650, the second device 2000 can display the chat window to show the conversation between the first user and the second user on the screen of the second device 2000, and in step S2655, the second device 2000 can receive an input. text by the second user through the chat window. In addition, the second device 2000 may arrange the text received from the first device 1000 and the text entered into the second device 2000 in a temporal order and may display the texts in the chat window.
In step S2660, the second device 2000 can convert the second user's text into a speech using text-to-speech (TTS). The second device 2000 may determine a tone of voice to be converted from the second user's text, in consideration of an age and gender of the second user.
In step S2665, the second device 2000 can transmit, to the first device 1000, the voice that is converted from the second user's text and the second user's text.
In step S2670, the first device 1000 can output the voice that is converted from the text of the second user, through the loudspeaker of the first device 1000.
Figure 27 is a flowchart of a method for conducting a voice message call, the method being performed by the first device 1000 operating in a voice mode and the second device 2000 operating in a voice mode. of text, according to an exemplary modality. In the embodiment of FIG. 27, the first device 1000 can convert a voice of a first user to a text and can convert a text of a second user to a voice. Consequently, the first user can operate in the voice mode, while the second user can operate in the text mode.
Since operations S2700 to S2755 of FIG. 27 correspond to operations S2600 to Ξ2655 of FIG. 26, the description of operations S2700 to S2755 is omitted here.
In step S2760, the second device 2000 can transmit a text that is entered by the second user (ie, the second user's text) to the first device 1000.
In step S2765, the first device 1000 can convert the second user's text into a voice. The first device 1000 can convert the second user's text into a second user's voice using various TTS techniques. In addition, the first device 1000 can obtain user information about a gender, an age, etc., in advance from the second user, and can determine a tone of voice to be converted from the second user's text, in consideration of gender, the age, etc., of the second user.
In operation S2770, the first device 1000 can output the voice that is converted from the text of the second user, through a loudspeaker of the first device 1000. Consequently, the second user can operate in the text mode, while the first user can operate in voice mode.
Figure 28 is a flow chart of a method for conducting a voice message call, the method being performed by the first device 1000 operating in a voice mode and the second device 2000 operating in a voice mode. of text, according to an exemplary modality. In the embodiment of FIG. 28, the second device 2000 can convert a voice of a first user to a text and can convert a text of a second user to a voice.
Since operations S2800 to S2835 of FIG. 28 correspond to operations S2700 to S2735 of FIG. 27, descriptions about operations S2700 to S2735 are omitted here.
In step S2840, the first device 1000 can transmit the voice of the first user to the second device 2000.
In operation S2845, the second device 2000 can convert the first user's voice to text, and in operation S2850, the second device 2000 can display text indicating a conversation between the first user and the second user in the window. of talk. Consequently, the first user can operate in the voice mode and the second user can operate in the text mode.
In step S2855, the second device 2000 can receive a text input from the second user through the chat window, and in step S2860, the second device 2000 can convert the text that is entered by the second user ( that is, the second user's text) in one voice. For example, the second device 2000 may determine a pitch of the converted voice, in consideration of a gender and an age of the second user.
In step S2865, the second device 2000 can transmit, to the first device 1000, the voice that is converted from the second user's text, the text that is converted from the first user's voice, and the second user's text.
In operation S2870, the first device 1000 can output the voice that is converted from the text of the second user, through a speaker of the first device 1000. Consequently, the first user can operate in the voice mode and the second user can operate in text mode.
Figures 29A, 29B and 29C illustrate examples of a combination mode which is one of the calling modes of a voice message call, according to empirical modalities.
During combination mode, while a voice input function and a voice output function for a voice call are activated, a text input function and a text output function can be activated for a text exchange. Thus, in the combination mode, a user can carry out text exchange while the user carries out the voice call through the first device 1000.
In addition, during combination mode, while at least one of the voice input function and the voice output function for the voice call are activated, at least one of the text input function and the voice call function can be activated. text output for exchanging text.
As an example, during merge mode, while the voice input function for voice call is activated, the text input function and the text output function for text exchange can be activated.
As another example, during merge mode, while the voice output function for voice call is activated, the text input function and text output function for text exchange can be activated.
As another example, during merge mode, while the voice input function and the voice output function for voice call are activated, the text input function for text exchange can be activated.
As another example, during merge mode, while the voice input function and the voice output function for the voice call are activated, the text output function for text exchange can be activated.
Referring to FIG. 29A, a loudspeaker function of the first device 1000 can be activated and a message exchange function can be activated through a chat window that is displayed on a screen of the first device 1000.
During combining mode, the first device 1000 can also carry out a message exchange with the second device 2000 while the first device 1000 carries out the voice call with the second device 2000.
Referring to Figure 29B, while activating a voice call function that does not involve the use of a loudspeaker function of the first device 1000, a message exchange function may be activated through a chat window that is displayed visually. on a screen of the first device 1000. The voice call function that does not involve the use of the loudspeaker function can indicate a voice call that is made using a loudspeaker adjacent to a user's ear and a microphone adjacent to a user's mouth, while the first device 1000 is adjacent to a face of the user.
In this case, while the first device 1000 carries out a voice input / output when the voice call function is activated, the first device 1000 can also carry out a text input / output through a conversation window to exchange a message with the second device 2000.
Referring to FIG. 29C, while a user uses a portable device 290 that is connected to the first device 1000 in a wired or wireless manner, the first device 1000 can perform a voice message call during a combination mode. In this case, the first device 1000 may control the portable device 290 to perform at least one of a voice input / output and a text input / output for the voice message call during combining mode.
For example, when the handheld device 290 is a headset, the first device 1000 may control a voice from the first device 1000 to be output through the headset while the first device 1000 is conducting the voice message call during talk mode. combination.
For example, when the wearable device 290 is smart glasses, the first device 1000 can control a chat window, which is broadcast from the first device 1000, to be broadcast through the smart glasses, as long as the first device 1000 performs the voice message call during merge mode.
FIG. 30 illustrates an example in which a call mode of the first device 1000 conducting a voice message call is changed from a voice mode to a combination mode or is changed from a combination mode to a combination mode. voice, according to an exemplary modality.
Referring to FIG. 30, when a button 300 displayed on a screen of the first device 1000 that performs the voice message call is selected during the voice mode, the call mode of the first device 1000 can be changed from the mode voice to combination mode. In addition, when a button 301 displayed on the screen of the first device 1000 that performs the voice message call is selected during the merge mode, the call mode of the first device 1000 can be switched from merge mode to merge mode. voice.
Fig. 31 illustrates an example in which a calling mode of the first device 1000 conducting a voice message call is changed from a text mode to a combination mode or is changed from a combination mode to a text mode. , according to an exemplary modality.
Referring to FIG. 31, when a button 310 displayed on a screen of the first device 1000 that performs the voice message call is selected during text mode, the call mode of the first device 1000 can be changed from the mode text to merge mode. Also, when a button 311 displayed on the screen of the first device 1000 that performs the voice message call is selected during merge mode, the call mode of the first device 1000 can be changed from merge mode to merge mode. text.
FIG. 32 is a flow chart of a method of changing a mode of a voice message call, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
Since operation S3200 to S3220, and operation S3260 to S3290 in the flow chart of FIG. 32 correspond to operation S2200 to S2220, and operation S2230 to S2260 of FIG. 22, the descriptions about operation S3200 through S3220, and operation S3260 through S3290 are ignored here.
As a result of the determination in step S3200, if a call mode is not a voice mode, in step S3230, the first device 1000 can determine whether the call mode of a voice message call is a combination mode. . For example, the first device 1000 can determine whether a first user selects a button to switch to the combination mode, and thus can determine whether the calling mode is the combination mode.
As a result of the determination in step S323 0, if it is determined that the calling mode is the combination mode, in step S3240, the first device 1000 can activate a speaker and a microphone in the first device 1000.
In step S3250, the first device 1000 may display a chat window on a screen of the first device 1000. Furthermore, the first device 1000 may display, in the chat window, a text that is entered by the first user to through the chat window. Furthermore, the first device 1000 may display, in the conversation window, a text that is converted from a second user's voice and a text that is entered to the second device 2000 by the second user.
As a result of the determination in step S3230, if it is determined that the calling mode is not the combining mode, the first device 1000 can carry out step S3260.
FIG. 33 is a flow chart of a method for carrying out a voice message call during a combining mode, the method being carried out by the first device 1000, in accordance with an exemplary mode.
In step S3300, the first device 1000 may determine that a calling mode of the voice message call is the combination mode. For example, the first device 1000 can determine whether a first user selects a button to switch to the combination mode, and thus can determine that the calling mode is the combination mode.
When the calling mode is the combination mode, in step S3310, the first device 1000 can activate a speaker and microphone, and in step S3320, the first device 1000 can display a chat window on a screen of the first device 1000. In addition, the first device 1000 may display, in the chat window, a text that is entered by the first user through the chat window. Furthermore, the first device 1000 may display, in the conversation window, a text that is converted from a second user's voice and a text that is entered to the second device 2000 by the second user.
In step S3330, the first device 1000 may receive a voice input by the first user and / or a text input by the first user. The first device 1000 may receive voice input by the first user through an activated microphone, and may receive text input by the first user through a visually displayed chat window. The first device 1000 can convert a voice that is entered by the first user into a text, or it can convert a text that is entered by the first user into a voice.
In step S3340, the first device 1000 can recognize a call mode of the second device 2000. The second device 2000 can transmit information indicating the call mode of the second device 2000 to the first device 1000 and the first device 1000 can acknowledge the call. of the second device 2000, based on the information indicating the call mode that is received from the second device 2000.
In step S3350, the first device 1000 can selectively transmit at least one of the first user's voice and text to the second device 2000. The first device 1000 can selectively transmit, to the second device 2000, at least one of the voice that is entered by the first user, the text that is entered by the first user, a text that is converted from the voice input by the first user and a voice that is converted from text input by the first user.
As an example, when the calling mode of the second device 2000 is a voice mode, the first device 1000 can transmit, to the second device 2000, the voice that is entered by the first user and the voice that is converted from the text entered by the first user. As another example, when the calling mode of the second device 2000 is a text mode, the first device 1000 can transmit, to the second device 2000, the text that is entered by the first user and the text that is converted from the voice entered by the first user. However, one or more exemplary modalities are not limited to that.
FIG. 34 is a flow chart of a method for conducting a voice message call with the second device 2000, the method being performed by the first device 1000 operating in a combination mode, in accordance with a exemplary mode.
In step S3400, the first device 1000 is connected by call with the second device 2000. The first device 1000 can transmit a call connection request to the second device 2000 or can receive a call connection request from the second device 2000, so that the first device 1000 can be call connected with the second device 2000.
In step S3405, the first device 1000 can determine whether a calling mode of the voice message call is a combination mode. For example, the first device 1000 can determine whether a first user selects a button to switch to the combination mode, and thus can determine whether the calling mode is the combination mode.
When the call mode is the combination mode, in step S3410, the first device 1000 can activate a speaker and microphone, and in step S3415, the first device 1000 can display a chat window on a screen of the first device 1000..
In step S3420, the first device 1000 can receive a voice input by the first user. The first device 1000 may receive the voice input by the first user through the activated microphone.
In step S3425, the first device 1000 can transmit the voice that is entered by the first user to the second device 2000, and in step S3430, the first device 1000 can convert the voice that is entered by the first user into a text. In addition, the first device 1000 may display, in the conversation window, the text that is converted from the voice input by the first user.
In step S3435, the second device 2000 can transmit a voice of a second user to the first device 1000, and in step S3440, the first device 1000 can broadcast the voice of the second user. The first device 1000 can output the voice of the second user through the activated loudspeaker.
In step S3445, the first device 1000 can convert the second user's voice into text. In addition, the first device 1000 can display, in the conversation window, the text that is converted from the second user's voice.
In step S3450, the first device 1000 can receive a text input by the first user. The first device 1000 can receive the text input by the first user through the chat window and can display the text entered by the first user in the chat window.
In step S3455, the first device 1000 can convert the text that is entered by the first user into a voice. For example, the first device 10 0 0 may determine a tone of voice to be converted from the text input by the first user, in consideration of a gender and an age of the first user. However, one or more exemplary modes are not limited thereto, and the pitch of the converted voice can be preset.
In step S3460, the first device 1000 can transmit, to the second device 2000, the voice that is converted from the text input by the first user.
Figure 3-5A is a flowchart of a method of advising a first user of a change in a calling mode of the second device 2000, and recommending a first user to change a calling mode, the method being carried out by the first device 1000, according to an exemplary embodiment.
In step S3500, the first device 100 can recognize a call mode of the second device 2000. The second device 2000 can transmit information indicating the call mode of the second device 2000 to the first device 1000 and the first device 1000 can acknowledge the call. of the second device 2000, based on the information indicating the call mode that is received from the second device 2000.
In step S3510, the first device 1000 can determine whether the calling mode of the second device 2000 is changed. For example, the call mode of the second device 2000 can be changed from a voice mode to a text mode, it can be changed from voice mode to a combination mode, it can be changed from text mode to voice mode, it can be changed from text mode to merge mode, it can be switched from merge mode to voice mode, or it can be switched from merge mode to text mode. Here, the first device 1000 may poll the second device 2000 for mode information. Alternatively, the second device 2000 may inform the first device 1000 of a mode change.
In addition, the first device 1000 may determine whether an input method of the second device 2000 is changed. The first device 1000 may determine whether the second device 2000 receives only one voice input, only one text input, or one voice and voice input. text for a voice message call. Again, the first device can poll the second device for information about the input method, or the second device 2000 can inform the first device about the input method.
In addition, the first device 1000 may determine whether an output method of the second device 2000 is changed. The first device 1000 may determine whether the second device 2000 provides only speech output, only text output, or speech and voice output. text for voice message call. The first device can poll the second device for information about the exit method, or the second device 2000 can inform the first device about the exit method.
As a result of the determination in step S3510, when the first device 1000 determines that the calling mode of the second device 2000 is changed, in step S3520, the first device 1000 can inform the first user of the changed calling mode of the second device . As an example, when a call mode of the first device 1000 is a voice mode, the first device 1000 may emit a voice to advise the change in the call mode of the second device 2000. As another example, when the call mode of the first device 1000 is a text mode, the first device 1000 can send, to a screen of the first device 1000, a text to advise the change in the call mode of the second device 2000. As another example, when the calling mode of the first device 1000 is a combination mode, the first device 1000 may output, to the display of the first device 1000, the text to advise the change in the calling mode of the second device 2000 while that the first device 1000 emits the voice to advise the change in the calling mode of the second device 2000. However, one or more exemplary modes are not limited thereto. In addition, the first device 1000 may advise the first user of an exit method during the changed call mode of the second device 2000.
Furthermore, when the input method of the second device 2000 is changed, the first device 1000 may inform the first user of the changed input method.
Furthermore, when the output method of the second device 2000 is changed, the first device 1000 may inform the first user of the changed output method.
In operation S3530, the first device 1000 may recommend a change in the calling mode of the first device 1000. As an example, while the first device 1000 operating in voice mode carries out a voice message call with the second device 2000, when the call mode of the second device 2000 changes from voice mode to text mode , the first device 10 0 0 may recommend to the first user to change the calling mode of the first device 1000 from voice mode to text mode. In this case, the first device 1000 can emit a voice. Do you want to change a call mode to a text mode? through a loudspeaker or message. Also, when the first device 1000 changes the calling mode from voice mode to text mode, the first device 1000 may temporarily cut off a voice call channel established between the first device 1000 and the second device 2000.
As another example, while the first device 1000 operating in the text mode makes a voice message call with the second device 2000, when the call mode of the second device 2000 is changed from text mode to voice mode , 1000 can recommend to the first user to change the call mode of the first device 1000 from text mode to voice mode. In this case, the first device 1000 can display a text Do you want to change a call mode to a voice mode? in a chat window on a screen of the first device 1000 or emit a voice message. When the first device 1000 changes the calling mode from text mode to voice mode, the first device 1000 may temporarily cut a channel for a text exchange to be established between the first device 1000 and the second device 2000.
For example, when the calling mode of the second device 2000 is changed, the first device 1000 may generate a vibration and may simultaneously display a message indicating a mode change on the screen of the first device 1000.
When the input method of the second device 2000 is changed, the first device 1000 may recommend an input method of the first device 1000 for conducting a voice message call. Furthermore, when the input method of the second device 2000 is changed, the first device 1000 may recommend an output method of the first device 1000 to carry out the voice message call.
Also, when the output method of the second device 2000 is changed, the first device 1000 may recommend an input method of the first device 100.0 for conducting a voice message call. Also, when the output method of the second device 2000 is changed, the first device 1000 may recommend an output method of the first device 1000 to carry out the voice message call.
Figure 35B is a flowchart of a method of advising a first user of a change in the input / output functions of the second device 2000 during a combination mode and recommending the first user to change a calling mode, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
In step S3540, the first device 1000 may recognize that the input / output functions of the second device 2000 are changed during the combining mode. The second device 2000 can transmit to the first device 1000 information indicating the change in the input / output functions of the second device 2000 during combining mode and the first device 1000 can recognize the change in the input / output functions of the second device. 2000 during merge mode, based on information received from the second device 2000.
In step S3550, the first device 1000 may determine whether the input / output functions of the second device 2000 are changed during the combining mode. For example, the first device 1000 may determine whether at least one of a voice input function and a voice output function is changed for a voice call or at least one of a text input function and an output function. text for a text exchange in the merge mode of the second device 2000.
In addition, the first device 1000 may determine whether an input method of the second device 2000 is changed. The first device 1000 may determine whether the second device 2000 receives only one voice input, only one text input, or one voice and voice input. text for a voice message call.
In addition, the first device 1000 may determine whether an output method of the second device 2000 is changed. The first device 1000 may determine whether the second device 2000 provides only speech output, only text output, or speech and voice output. text for voice message call.
As a result of the determination in step S3550, when the first device 1000 determines that the input / output functions of the second device 2000 during the combining mode are changed, in step S3560, the first device 1000 can inform the first user of the modified input / output functions of the second device 2000. For example, the first device 1000 may inform the first user of a changed output method of the second device 2000. Also, when the input method of the second device 2000 is changed, the first device 1000 may inform the first user of a method of output. input changed from the second device 2000. When the output method of the second device 2000 is changed, the first device 1000 may inform the first user of a changed output method of the second device 2000.
In operation S3570, the first device 1000 may recommend a change in the calling mode of the first device 1000. In this case, the first device 1000 may recommend to the first user a calling mode of the first device 1000, according to the change in the input / output functions of the second device 2000 during combination mode.
When the input method of the second device 2000 is changed, the first device 1000 may recommend an input method of the first device 1000 for conducting a voice message call. Furthermore, when the input method of the second device 2000 is changed, the first device 1000 may recommend an output method of the first device 1000 to carry out the voice message call.
Furthermore, when the output method of the second device 2000 is changed, the first device 1000 may recommend an input method of the first device 1000 for conducting a voice message call. Also, when the output method of the second device 2000 is changed, the first device 1000 may recommend an output method of the first device 1000 to carry out the voice message call.
Figure 36 illustrates an example in which the first device 1000 recommends a first user to change a calling mode from a voice mode to a text mode, in accordance with an exemplary mode.
Referring to FIG. 36, the first device 1000 may carry out a voice message call as long as the first device 1000 operates during the voice mode and then when a call mode of the second device 2000 is changed from a voice mode to a text mode, you can emit a voice The call mode on the other side is changed from a voice mode to a text mode. Do you want to change your calling mode to a text mode? through a speaker of the first device 1000.
In addition, the first user can have the first device 1000 away from a face of the first user, and therefore the first device 1000 can change the calling mode from voice mode to text mode.
FIG. 37 illustrates an example in which the first device 1000 recommends a first user to change a calling mode from a text mode to a voice mode, in accordance with an exemplary mode.
Referring to FIG. 37, the first device 1000 may carry out a voice message call as long as the first device 1000 is operating during text mode and then when a call mode of the second device 2000 is changed from a text mode to a voice mode, You can emit a voice The call mode on the other side is changed from a text mode to a voice mode. Do you want to change your calling mode to a voice mode? through a speaker of the first device 1000.
Furthermore, the first user can make the first device 1000 adjacent to a face of the first user, and thus the first device 1000 can change the calling mode from text mode to voice mode.
Fig. 3-8 is a flow chart of a method for generating a call list in relation to voice message calls, the method being carried out by the first device 1000, in accordance with an exemplary embodiment.
In step S3800, the first device 1000 can obtain device information about the second device 2000 that carries out a voice message call with the first device 1000. For example, the first device 1000 can obtain information about a service set identifier (SSID) of the second device 2000, a phone number of the second device 2000, a name of a second user of the second device 2000, and a type of the second device 2000.
• In operation S3810, the first device 1000 can obtain information about a time in relation to a call with the second device 2000. For example, the first device 1000 can get information about a time when the first device 1000 and the second device 2000 initiate the voice message call, a time when the first device 1000 and the second device 2000 end the call. of voice message and a period of time during which the first device 1000 and the second device 2000 carry out the voice message call.
In step S3820, the first device 1000 can obtain information about a calling location from the first device 1000. The first device 1000 can obtain information about a location from the first device 1000 while the first device 1000 and the second device 2000 carry out the voice message call.
In step S3830, the first device 1000 can back up text information about a text in a chat window generated during the voice message call. The first device 1000 may back up the text in the chat window indicating a conversation between the first user and the second user. The text can be backed up on device 1000 or on a server that mediates the connection between device 1000 and device 2000.
In addition, the first device 1000 can back up a text that is obtained for a preset period of time before a predetermined event occurs. For example, the default event may include an end of a call between the first device 1000 and the second device 2000, receipt of user input for a text backup, or the like. For example, when the call between the first device 1000 and the second device 2000 ends, the first device 1000 can back up a text that is obtained by the first device 1000 for five minutes before the call ends. For example, when the first device 1000 receives user input for the text backup, the first device 1000 can back up text that is obtained by the first device 1000 for three minutes before user input is received. However, an event type and a time value that are set for text backup are not limited to this.
Also, the first device 1000 can back up a part of the text indicating the conversation between the first user and the second user on the first device 1000 and can back up all the text indicating the conversation between the first user and the second user on the server. 3000.
In step S3840, the first device 1000 can generate the call list, based on a plurality of pieces of information obtained in steps S3800 to S3820.
In step S3850, the first device 1000 can associate the generated call list with the text information that is backed up in step S3830 and can store the call list. Because the call list is associated with the backed-up text information and is then stored, the first device 1000 can provide the first user with a conversation content that corresponds to the call list, based on user input regarding with the call list. In this case, when a first button is selected from the call list, the first device 1000 may first display a portion of the conversation content corresponding to the call list on a screen of the first device 1000 and when a second device is selected button on the screen showing the part of the conversation content, The first device 1000 can display all the conversation content that corresponds to the call list on the screen of the first device 1000.
FIG. 39 is a flow diagram of a text display method indicating a conversation between a first user and a second user through a call list of the first device 1000, in accordance with an exemplary embodiment.
In operation S3900, the first device 1000 can select the second device 2000 from the call list of the first device 1000. The first device 1000 can select a phone number of the second device 2000 or a name of the second user from the list of calls. calls, based on user input.
In step S3910, according to the selection of the second device 2000, the first device 1000 may display the text indicating the conversation between the first user and the second user on a screen of the first device 1000.
Figures 40 to 42 illustrate examples of a call list, according to exemplary embodiments.
Referring to Figure 40, the item numbers
100 telephone of a plurality of the second devices 2000 can be displayed in the call list that is displayed on a screen of the first device 1000. In addition, to view the conversation content related to a specific call, a first user can select a button 350 in the call list. In this sense, a user can see a history of the conversation with another user.
When the first user selects the button 350, a text indicating the conversation content corresponding to the selected button 350 may be displayed on an entire portion of the screen of the first device 1000, as illustrated in Figure 41. In this case, the Text indicating the content of the conversation may correspond to a part of the content of the conversation. Furthermore, the text indicating the content part of the conversation may be stored in the first device 1000, but one or more exemplary embodiments are not limited thereto.
Also, referring to FIG. 41, a button 410 may be displayed to display the entire content of the conversation in a window that includes the text indicating the content portion of the conversation. When the first user presses button 410, the first device 1000 can display text indicating all content.
101 conversation on the screen of the first device 1000. In this case, the text indicating all the conversation content can be stored in the server 3000, and the first device 1000 can receive the text indicating all the conversation content from the server 3000 and can visually present the text.
When the first user selects the button 350, a pop-up window including text indicating the conversation content corresponding to the selected button 350 can be displayed on a portion of the screen of the first device 1000, as shown in Figure 42.
When the first user selects the button 350, all the conversation content corresponding to the selected button 350 may be displayed, but one or more exemplary embodiments are not limited thereto.
FIG. 43 illustrates an example in which the first device 1000 and the second device 2000 carry out a voice message call through the server 3000, according to an exemplary embodiment.
Referring to Fig. 43, the first device 1000 can carry out the voice message call with the second device 2000 through the server 3 000. In addition, the server 3 00 0 can carry out a TTS function and a STT for message call
102 voice
For example, the server 3000 may convert the voice of a first user entered through the first device 10 00 into text and the text of a first user entered through the first device 1000 into a voice. Furthermore, the server 3000 may convert the voice of a second user that is entered through the second device 2000 into a text and the text of a second user that is entered through the second device 2000 into a voice.
Figure 44 is a. flow chart of a method of supporting a voice message call between the first device 1000 and the second device 2000 by converting the voices of the first and second users into texts, the method being carried out by the server 3000, according to with an exemplary modality.
In step S4400, the first device 1000 and the second device 2000 are connected to each other. The first device 1000 may be call connected to the second device 2000 by transmitting a call connection request to the second device 2000 or by receiving a call connection request from the second device 2000.
In step S4405, the first device 1000 may activate a voice message call function. He
103 The first device 1000 can activate the voice message call feature, based on user input. In step S4405, the first device 1000 activates the voice message calling function after the first device 1000 is connected by call with the second device 2000. However, one or more exemplary embodiments are not limited thereto. For example, the first device 1000 may activate the voice message call function when the first device 1000 sends a call or receives a call or while the first device 1000 carries out a call.
In step S4410, the second device 2000 may activate a voice message calling function. The second device 2000 may activate the voice message call function, based on user input. In step S4410, the second device 2000 activates the voice message calling function after the second device 2000 is connected by call with the first device 1000. However, one or more exemplary embodiments are not limited thereto. For example, the second device 2000 may activate the voice message call function when the second device 2000 sends a call or receives a call or while the second device 2000 carries out a call.
In step S4415, the first device 1000
104 it can transmit a voice of the first user to the server 3000. The first device 1000 can obtain the voice of the first user through a microphone in the first device 1000 and can transmit the obtained voice to the server 3000.
In step S4420, the server 3000 can transmit the voice of the first user to the second device 2000.
In step S4425, the server 3000 can convert the voice of the first user to text. Server 3000 can convert the first user's voice to text using various STT techniques.
In step S4430, the server 3000 can transmit the text that is converted from the voice of the first user to the second device 2000, and in step S4435, the server 3000 can transmit the text that is converted from the voice of the first user to the first device 1000.
In step S4440, the second device 2000 can transmit the voice of the second user to the server 3000. The second device 2000 can obtain the voice of the second user through a microphone in the second device 2000 and can transmit the obtained voice to the server 3000 .
In step S4445, the server 3000 can transmit the second user's voice to the first device 1000.
In operation S4450, the server 3000 can
105 convert the second user's voice to text. The server 3000 can convert the second user's voice to text using various STT techniques.
In operation S4455, the server 3000 can transmit the text that is converted from the second user's voice to the first device 1000, and in operation S4460, the server 3000 can transmit the text that is converted from the second user's voice to the second. device 2000.
In step S4465, the first device 1000 can arrange the received texts in a temporal order. As an example, according to a time when the voice of the first user is input to the first device 1000 and a time when the voice of the second user is received, the first device 1000 can arrange the text to be converted from the voice of the first user and the text that is converted from the voice of the second user. As another example, according to the time when the voice of the first user is entered into the first device 1000 and a time when the voice of the second user is entered into the second device 2000, the first device 1000 can arrange the text that be converted from the voice of the first user and the text to be converted from the voice of the second user. However, one or more exemplary modalities are not limited to that.
In step S4470, the first device 1000
106 can display a chat window that includes the texts arranged on a screen of the first device 1000. The first device 1000 can display the chat window on the screen of the first device 1000 while the first device 1000 conducts the call from voice. Furthermore, the first device 1000 may display the chat window on the screen of the first device 1000 or it may hide the chat window on the screen of the first device 1000, depending on the calling mode of the voice message call.
In step S4475, the second device 2000 can arrange the received texts in a temporal order. As an example, according to a time when the second user's voice is input to the second device 2000 and a time when the first user's voice is received, the second device 2000 can arrange the text to be converted from the voice of the first user and the text that is converted from the voice of the second user. As another example, according to the time when the second user's voice is input to the second device 2000 and a time when the first user's voice is input to the second device 2000, the second device 2000 can arrange the text that be converted from the voice of the first user and the text that is converted from the voice of the second user. Without
107 However, one or more exemplary modalities are not limited to that.
In step S4480, the second device 2000 can display a chat window that includes the texts arranged on one screen of the second device 2000. The second device 2000 can display the chat window on the screen of the second device 2000 while the second device 2000 make the voice call. Furthermore, the second device 2000 may display the chat window on the second device 2000 screen or may hide the chat window on the second device 2000 screen, depending on the calling mode of the voice message call.
As explained above, the server 3000 can mediate the connection between the first device 1000 and the second device 2000. Accordingly, the server 3000 can perform the conversion between text and speech as needed. In addition, the server can carry out storage functions, thus storing text and voice data as required.
Figure 4-5 is a flow chart of a method of supporting a voice message call between the first device 1000 and the second device 2000 by converting the texts of the first and second users into voices,
108 the method being carried out by the server 3000, according to an exemplary embodiment.
Because operations S4500 to S4510 of FIG. 45 correspond to operations S4400 to S4410 of FIG. 44, their descriptions are omitted here.
In step S4515, the first device 1000 can transmit the text that is entered by the first user to the server 3000, and in step S4520, the server 3000 can transmit the text that is entered by the first user to the second device 2000.
In step S4525, the server 3000 can convert the text that is entered by the first user into a voice. The server 3000 may determine a tone of voice to be converted from the text of the first user, taking into account the user's information (eg, a gender and an age) about the first user.
In operation S4530, the server 3000 can transmit, to the second device 2000, the voice that is converted from the text of the first user, and in operation S4535, the server 3000 can transmit, to the first device 1000, the speech that is Convert from the text of the first user.
In step S4540, the second device 2000 can transmit the text that is entered by the second user to the server 3000, and in step S4545, the
109 Server 3000 can transmit the text that is entered by the second user to the first device 1000.
In step S4550, the server 3000 can convert the text that is entered by the second user into a voice. The server 3000 may determine a tone of voice to be converted from the second user's text, in consideration of user information (eg, a gender and an age) about the second user.
In step S4555, the server 3000 can transmit, to the first device 1000, the voice that is converted from the second user's text, and in step S4560, the server 3000 can transmit, to the second device 2000, the voice that is converted. Convert from the second user's text.
In step S4565, the first device 1000 may display a chat window showing a conversation between the first user and the second user, depending on the state of the first device 1000. For example, when a calling mode of the first device 1000 is a text mode or a combination mode, the first device 1000 can display the chat window showing the conversation between the first user and the second user on the screen of the device. first device 1000.
In step S4570, the first device 1000
110 can output the voice of the second user, depending on the state of the first device 1000. For example, when the call mode of the first device 1000 is a voice mode or the combination mode, the first device 1000 can output the voice of the second user .
In step S4575, the second device 2000 may display a chat window showing a conversation between the first user and the second user, depending on the state of the second device 2000. For example, when a calling mode of the second device 2 00 0 is a text mode or a merge mode, the second device 2000 may display the chat window showing the conversation between the first user and the second user in the screen of the second device 2000.
In step S4580, the second device 2000 may output the voice of the first user, depending on the state of the second device 2000. For example, when a call mode of the second device 2000 is a voice mode or the combination mode, the second Device 2000 can output the voice of the first user.
As explained above, the server 3000 can mediate the connection between the first device 1000 and the second device 2000. Accordingly, the server 3000 can perform the conversion between text and
111 voice as needed. In addition, the server can carry out storage functions, thus storing text and voice data as required.
Fig. 46 is a flow chart of a method of storing voice data and text data related to a voice message call between the first device 1000 and the second device 2000, the method being carried out by the server 3000 , according to an exemplary modality.
In operation S4600, the first device 1000 can carry out the voice message call with the second device 2000. The first device 1000 and the second device 2000 can carry out the voice message call through the server 3000, but one or more exemplary modalities are not limited to that. The first device 1000 can directly carry out the voice message call with the second device 2000.
In step S4610, the first device 1000 can store a text indicating a part of a conversation between users. The first device 1000 can get a text that is converted from a voice that a first user enters in the first device 1000, a text that is converted from a voice that a second user enters the second device 2000, a text that the first user enters the first device 1000, and a text that the
112 second user enter 2000 on the second device.
Furthermore, the first device 1000 can obtain a text that indicates the conversation between users of at least one of the first device 1000, the second device 2000 and the server 3000. Furthermore, the first device 1000 can store a part of the text obtained in the first device 1000.
In operation S4620, the first device 1000 can transmit a text indicating all the conversation between users to the server 3000, and in operation S4630, the server 3000 can store the text indicating all the conversation between users. In operations S4620 and S4630, the server 3000 receives the text indicating all the conversation between users from the first device 1000 and stores the conversation, but one or more exemplary modes are not limited thereto. The server 3000 can receive the text indicating a part of the conversation between users of the first device 1000, it can receive a text that indicates a part of the conversation between users of the second device 2000 and can directly generate a text that indicates a part of the conversation between users. Likewise, the server 3000 can obtain the text that indicates the entire conversation between users by combining the text received from the first device 1000, the text received from the second device 2000 and the text generated by the
113 server 3000.
In step S4640, the first device 1000 can store voice data indicating the text indicating the part of the conversation between users. The first device 1000 can get voice data that is converted from a text that the first user enters the first device 1000, voice data that is converted from a text that the second user enters the second device 2000, voice data that the first user enters the first device 1000, and voice data that the second user enters the second device 2000.
In addition, the first device 1000 can obtain voice data indicating the conversation between users of at least one of the first device 1000, the second device 2000 and the server 3000. In addition, the first device 1000 can store a part of the voice data obtained in the first device 1000.
In step S4650, the first device 1000 can transmit voice data indicating all conversation between users to server 3000, and in step S4660, server 3000 can store voice data indicating all conversation between users. In operation S4650 and S4660, the server 3000 receives the voice data indicating all the conversation between users from the first device 1000 and stores it, but one or more
114 Exemplary modalities are not limited to that. The server 3000 can receive voice data indicating a part of the conversation between users from the first device 1000, can receive voice data indicating a part of the conversation between users from the second device 2000, and can directly generate voice data indicating a part of the conversation between users. In addition, the server 3 000 can obtain the voice data indicating all the
<td>conversation</td><td>between users combining the</td><td>data</td><td>of</td><td>voice</td>
<td>received from</td><td>first device 1000, the</td><td>data</td><td>of</td><td>voice</td>
<td>received from</td><td>second device 2000 and</td><td>data</td><td>of</td><td>voice</td>
<td colspan="2">generated by server 3000. · Figure 47 illustrates in the example</td><td>at</td><td>what</td><td>he</td>
The first device 1000, the second device 2000, and a third device 4000 carry out a voice message call with each other, according to an exemplary embodiment.
Referring to FIG. 47, the voice message call may take place between the first device 1000, the second device 2000, and the third device 4000. For example, as long as the first device 1000 and the second device 2000 perform a voice message call, the third device 4000 may participate in the voice message call carried out by the first device 1000 and the second device 2000. Alternatively, for example, the first device
115
1000, the second device 2000 and the third device 4000 may jointly initiate a voice message call. Although three devices are illustrated, any number of devices can participate in a voice message call.
When the voice message call is carried out between the first device 1000, the second device 2000 and the third device 4000, i) one or more operations between the first device 1000 and the second device 2000 to carry out the call from voice message, ii) one or more operations between the first device 1000 and the third device 4000 to carry out the voice message call, and iii) one or more operations between the second device 2000 and the third device 4000 to carry out the voice message call may correspond to the operations illustrated and described with reference to Figures 1 to 42.
Figure 48 illustrates an example in which the first device 1000, the second device 2000, and the third device 4000 carry out a voice message call with each other through the server 3000, in accordance with an exemplary embodiment.
Referring to Figure 48, when the first device 1000, the second device 2000 and the third device 4000 carry out the message call
116 voice input to each other through the server 3 000, a voice input to the first device 1000, a voice input to the second device 2000, and a voice input to the third device 4000 can be fully or partially converted in texts by server 3000. Also, a text that is entered in the first device 1000, a text that is entered in the second device 2000 and a text that is entered in the third device 4000 can be totally or partially converted into voices by the server 3000.
Furthermore, when the first device 1000, the second device 2000 and the third device 4000 carry out the voice message call with each other through the server 3000, i) one or more operations between the first device 1000, the second device 2000 and the server 3000 to carry out the voice message call, ii) one or more operations between the first device 1000, the third device 4000 and the server 3000 to carry out the voice message call, and iii) one or more operations between the second device 2000, the third device 4000, and the server 3000 to carry out the voice message call may correspond to the operations illustrated and described with reference to Figures 43 to 46.
Figures 4, 9 and 50 are block diagrams of the first device 1000, according to modalities of the
117 present invention.
As illustrated in Figure 49, the first device 1000 may include a user input unit 1100, an output unit 1200, a control unit 1300, and a communication unit 1500. However, the first device 1000 may be carried out with more or fewer elements than the elements shown in figure 49.
For example, as illustrated in Figure 50, the first device 1000 may further include a detection unit 1400, an audio / video (A / V) input unit 1600, and a memory 1700 in addition to the user input unit. 1100, output unit 1200, control unit 1300, and communication unit 1500.
The user input unit 1100 may be a unit by which a user enters data to control the first device 1000. For example, user input unit 1100 may include a keyboard, a dome switch, a touch pad, a pressure-resistant type touch pad, an infrared beam detection type touch pad, an infrared beam detection type touch pad, and a touch pad. surface acoustic wave type, an integral extensometric type touch pad, a piezoelectric effect type touch pad or the like), but one or more exemplary modalities are not limited to
118 that.
User input unit 1100 may receive user input for a voice message call. For example, user input unit 1100 may receive voice input from a user, text input from a user, and touch input from a user.
The output unit 1200 may function to output an audio signal, a video signal, or a vibration signal, and may include a display unit 1210, a sound output unit 1220, a vibration motor 1230, or the like.
The display unit 1210 may display and output information that is processed by the first device 1000. For example, the display unit 1210 may display a chat window that includes text indicating a conversation between users during the call of voicemail.
When the display unit 1210 and a touch pad form a mutual layer structure and then are formed as a touch screen, the display unit 1210 can be used as both an output device and an input device. The display unit 1210 may include at least one of a liquid crystal display (LCD), a thin-film transistor-liquid crystal display (TFT-LCD), a display
119 organic light-emitting diode, a flexible screen, a three-dimensional screen and an electrophoretic screen. Furthermore, according to one type of the first device 1000, the first device 1000 may include at least two display units 1210. Here, the at least two display units 1210 may face each other using a hinge.
The sound output unit 1220 can output audio data that is received from the communication unit 1500 or stored in memory 1700. In addition, the sound output unit 1220 outputs a sound signal (for example, a reception of a ring signal, an alarm sound or the like, related to the capabilities carried out by the first device 1000. The sound output unit 1220 may include a speaker, a buzzer or the like.
The vibration motor 1230 can output a vibration signal. For example, the vibration motor 1230 may output the vibration signal that corresponds to an output of the audio data (for example, the call signal reception sound, the message reception sound, or the like) or data from video. In addition, when a touch is input via a touch screen, the vibration motor 1230 can output a vibration signal.
Control unit 1300 can generally control all operations of the first device
120
1000. For example, the control unit 1300 can control the user input unit 1100, the output unit 1200, the detection unit 1400, the communication unit 1500, the A / V input unit, the output unit 1200 , the detection unit 1400, the communication unit 1500, the A / V input unit 1600, etc., to carry out operations of the first device 1000 in Figures 1 to 49.
In more detail, the control unit 1300 can activate a voice message call function. When the first device 1000 sends a call to the second device 2000, the control unit 1300 can activate the voice message call function. Alternatively, when the first device 1000 receives a call from the second device 2000, the control unit 1300 can activate the voice message call function. Alternatively, while the first device 1000 carries out a call with the second device 2000, the control unit 1300 can activate the voice message call function.
In addition, when the voice message call function is activated, the control unit 1300 may inform the second device 2000 that the voice message call function is activated in the first device 1000. In this case, the control unit 1300 can advise the second
121 device 2000 of a calling mode of the voice message call of the first device 1000.
The control unit 1300 can obtain texts that are converted from a voice of a user of the first device 1000 and a voice of a user of the second device 2000. The control unit 1300 can convert the voice of the user of the first device 1000 into a text and thus you can get the converted text. In addition, the control unit 1300 can receive the voice of the user of the second device 2000 from the second device 2000 through the communication unit 1500, can convert the voice to a text, and thus can get the converted text. However, a method of obtaining the converted text, the method being carried out by the first device 1000, is not limited thereto. The voice of the user of the first device 1000 and the voice of the user of the second device 2000 can be converted into texts by various devices or a server and the first device 1000 can receive the converted texts from the various devices and the server through the unit. communication 1500.
The control unit 1300 can obtain a text that is entered by a user. The control unit 1300 can obtain a text that is entered by the user of the first device 1000 during a call of
122 voicemail. Furthermore, during the voice message call, the control unit 1300 may receive, from the second device 2000, a text that is entered into the second device 2000 by the user of the second device 2000.
The control unit 1300 can arrange the converted texts and the input texts in a temporal order and can display the converted texts and the input texts on a screen of the first device 1000 during a call between users. The control unit 1300 can arrange the converted texts and input texts in temporal order, based on the input times of the voices and texts. For example, based on a time when the user of the first device 1000 enters the voice in the first device 1000, a time that the user of the second device 2000 enters the voice in the second device 2000, a time that the user of the first device 1000 enters the text in the first device 1000 and a time when the user of the second device 2000 enters the text in the second device 2000, the control unit 1300 can arrange the converted texts and the input texts.
Control unit 1300 can back up original voice data from converted texts in memory 1700 or server 3000. Control unit 1300 can
123 backing up first user's voice data that corresponds to the text that is converted from the first user's voice. In addition, the control unit 1300 can back up second user's voice data corresponding to the text that is converted from the second user's voice.
The control unit 1300 can display a chat window on the screen of the first device 1000 during a call between the first user and the second user. The control unit 1300 may display the chat window on the screen of the first device 1000 to display the content of a conversation between the first user and the second user.
The control unit 1300 can obtain a text that is entered by the first user through the chat window. In addition, the control unit 1300 can receive, from the second device 2000, a text that the second user enters to the second device 2000 during the voice message call.
The control unit 1300 can arrange the converted texts and input texts in a temporal order and can display the converted texts in the conversation window. The control unit 1300 can arrange the converted texts and the input texts in the temporal order, based on the input times of the
124 voices and texts. For example, the control unit 1300 may arrange the converted texts and the input texts, based on a time when the first user enters the voice in the first device 1000, a time when the second user enters the voice in the second. device 2000, when the first user enters the text in the first device 1000 and a time when the second user enters the text in the second device 2000.
The control unit 1300 can distinguish between the converted texts and the input texts and can present them visually. The control unit 1300 can distinguish the converted text of the first user's voice and the converted text of the second user's voice from the text input by the first user and the text entered by the second user and can display them.
In order to distinctively display the converted texts, the control unit 1300 may display a separate icon around the converted texts. Alternatively, the control unit 1300 may distinctively display a color, thickness, or font of the converted texts. However, one or more exemplary modalities are not limited to that.
When at least one converted text is selected among the converted texts, the control unit 1300 can
125 output original voice data of the selected converted text. When the first user selects the converted text in the conversation window, the control unit 1300 can extract the original voice data of the selected converted text from a memory and can output the extracted original voice data through a loudspeaker in the first device 1000.
The control unit 1300 can back up a text between arranged texts, wherein the text is obtained for a preset period of time before a predetermined event occurs. For example, the default event may include an end of a call between the first device 1000 and the second device 2000, receipt of user input for a text backup, or the like. As an example, when the call between the first device 1000 and the second device 2000 ends, the first device 1000 can back up a text that is obtained by the first device 1000 for five minutes before the call ends. As another example, when the first device 1000 receives input from the user for the text backup, the control unit 1300 can back up a text that is obtained by the first device 1000 for three minutes before the input from the text is received. user. However, an event type and a time value that are set to
126 text backup is not limited to that.
The control unit 1300 can support all the texts arranged in the server 3000. In this case, the control unit 1300 can support some of the texts arranged in the first device 1000 and can support all the texts arranged in the server 3000. As Alternatively, the first device 1000 may back up some of the laid out texts and all of the laid out texts on the server 3000.
The control unit 1300 can analyze a conversation between users by performing various natural language-based analysis methods on all arranged texts and can extract a portion of arranged texts indicating important contents of the conversation between users. Furthermore, the control unit 1300 can support a text indicating the extracted part of the conversation in the first device 1000.
The control unit 1300 can activate a voice message call function and can determine a chat application to be used during a voice message call. As an example, the control unit 13 00 may determine a chat application that is executable by interoperating with the voice message call function of the first device 1000, such as the chat application to be used during the voice message call. As another
127 For example, the control unit 1300 may determine a chat application that is installed throughout the first device 1000 and the second device 2000, as the chat application to be used during the voice message call. However, one or more exemplary modalities are not limited to that.
The control unit 1300 can execute the determined chat application and can enter a text that is converted from a speech in the executed chat application. For example, the control unit 1300 may install a plug-in for automatic text input in the running chat application and may input the converted text of the speech into the chat application through the installed plug-in. However, one or more exemplary modalities are not limited to that.
Control unit 1300 may display a chat window on the screen of the first device 1000 during a call between users. The control unit 1300 can arrange, in the conversation window, a text that is converted from a voice of the first user, a text that is entered in the first device 1000 by the first user, a text that is converted from of a voice of the second user and a text that is input to the second device 2000 by the second user.
128
Furthermore, the control unit 1300 may display the chat window on the screen of the first device 1000 or it may hide the chat window on the screen of the first device 1000 depending on a calling mode of the voice message call of the first device. device 1000.
The control unit 1300 can initiate a voice message call while the first device 1000 and the second device 2000 use a chat service.
The control unit 1300 can recognize that a call mode of the first device 1000 is changed during the voice message call. The call mode can include a voice mode, a text mode, and a combination mode. When the call mode is changed, the control unit 1300 can activate or deactivate at least one of the speakers and a microphone of the first device 1000. In addition, when the call mode is changed, the control unit 1300 can display or hide the conversation window used in calling voice messages on the screen of the first device 1000. In addition, the control unit 1300 can selectively provide a voice or text from a user to the second device 2000, depending on a call mode of the second device 2000.
The control unit 1300 can inform the first user of a change in the call mode of the second
129 device 2000 and can recommend to the first user to change the calling mode of the first device 1000. In addition, the control unit 1300 can advise the first user of a change in the input / output functions of the second device 2000 during a combination mode , and can recommend to the first user to change the call mode of the first 1000 device.
The control unit 1300 can back up some or all of the text information about a text in the chat window on the first device 1000 or the server 3000. The control unit 1300 can associate the text information in the chat window with a call list and can store the call list.
The detection unit 1400 can detect the state of the first device 1000 or an environmental state around the first device 1000 and can supply information about the detected states to the control unit 1300.
Sensing unit 1400 may include at least one of a magnetic sensor 1410, an acceleration sensor 1420, a temperature / humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, a position sensor (eg, GPS) 1460, a 1470 air pressure sensor, a 1480 proximity sensor, and a 1490 RGB sensor (that is, a luminance sensor), but one or more
130 Embodiments of the present invention are not limited thereto. The functions of the sensors can be intuitively deduced by a person skilled in the art by referring to the names of the sensors, therefore a detailed description of the sensors is omitted here.
Communication unit 1500 may include one or more elements that allow first device 1000 to communicate with at least one of second device 2000, third device 4000, and server 3000. For example, communication unit 1500 may include a communication unit. short-range communication 1510, a mobile communication unit 1520, and a broadcast receiving unit 1530.
The short-range communication unit 1510 may include, but is not limited to, a Bluetooth communication unit, a BLE communication unit, an NFC / RFID unit, a Wi-Fi communication unit, a ZigBee communication unit, a IrDA communication unit, a WFD communication unit, a UWB communication unit, an Ant + communication unit, or the like.
Mobile communication unit 1520 transmits and receives a wireless signal with at least one of a base station, an external terminal, and a server in a mobile communication network. The wireless signal can include various types of data according to the communication of a
131 Sound call sign, a video call sign, or a text / multimedia message.
The broadcast receiver unit 1530 receives a broadcast signal and / or broadcast-related information from the outside through a broadcast channel. The broadcast channel can include a satellite channel and a ground wave channel. According to one embodiment, the first device 1000 may not include the broadcast receiving unit 1530.
The communication unit 1500 may allow the first device 1000 to exchange information for a voice message call with at least one of the second device 2000, the third device 4000, and the server 3000.
The A / V input unit 1600 may receive an input of an audio signal or a video signal and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image image such as a still image or a picture. moving through an image during a video call mode or an image capture mode. An image that is captured through the image sensor can be processed by the control unit 13 0 0 or a separate image processing unit (not shown).
The image frame that is processed by camera 1610 can be stored in memory 1700 or can be
132 transmitted to an external source through the communication unit 1500. According to the configuration of the first device 1000, two or more cameras 1610 can be arranged.
Microphone 1620 receives an external sound signal as input and processes the received sound signal into electrical voice data. For example, the microphone 1620 can receive a sound signal from an external device or a speaker. In order to eliminate noise that occurs while the sound signal is being input externally, the 1620 microphone can use various denoising algorithms.
Memory 1700 may store a program for processing and controlling control unit 1300, or it may store a plurality of pieces of data that are input to the first device 1000 or output from the first device 1000.
The memory 1700 may include a storage medium for at least one type of flash memory, a hard disk, a memory type multimedia card, a memory type card such as an SD or XD card memory, a random access memory (RAM ), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable only memory
133 reading (PROM), a magnetic memory, a magnetic disk and an optical disk.
The programs stored in memory 1700 can be classified into a plurality of modules according to their functions, for example, in a user interface (UI) module 1710, a touch screen module 1720, an alarm module 1730, etc.
User interface module 1710 may provide a specialized user interface or graphical user interface (GUI) in relation to the first device 1000 for each application. The touch screen module 1720 can detect the touch gesture of a user on a touch screen and transmit information related to the touch gesture to the control unit 1300. The touch screen module 1720 can recognize and analyze a contact code. The 1720 touch screen module can be configured using additional hardware that includes a controller.
Various sensors can be arranged on or near the touch screen to detect a touch or a close touch on the touch sensor. An example of the sensor for detecting touch on the touch screen may include a touch sensor. The touch sensor detects a contact from a specific object with at least the same sensitivity that a person can detect. The touch sensor can detect various types of
134 information such as the roughness of a contact surface, the hardness of the contact object, the temperature of a contact point or the like.
An example of the sensor for detecting touch on the touch screen may include a proximity sensor.
The proximity sensor detects the existence of an object that approaches a predetermined sensing surface or that exists nearby, using a force of an electromagnetic field or an infrared beam, instead of a mechanical contact. Examples of the proximity sensor include a transmission type photoelectric sensor, a direction reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacity proximity sensor, a magnetic proximity sensor. , an infrared type proximity sensor, or the like. The user's touch gesture (i.e. input) can include a tap gesture, a tap and hold gesture, a double turn gesture, a drag gesture, a scroll gesture, a flick gesture, a flick gesture. drag and drop, or the like.
The alarm module 173 0 can generate a signal to alarm the occurrence of an event. Examples of the event occurring on the first device 1000 may include a ring signal reception, a message reception, a key signal input, a schedule notification
135 or similar. The 1730 alarm module can output a video format alarm signal through the 1210 display unit, it can output an audio format alarm signal through the 1220 sound output unit or a vibration signal through vibration motor 1230.
FIG. 51 is a block diagram of the server 3000, according to an exemplary embodiment.
Referring to Figure 51, the server 3000 may include a communication unit 3100, a control unit 3200, and a database (DB) 3300. The DB 3300 may include a voice DB 3310, a text DB 3320, and a 3330 call list DB.
Communication unit 3100 can transmit data that is required to carry out a voice message call between at least two of the first device 1000, the second device 2000, and the third device 4000, to at least two of the first device 1000. , the second device 2000, and the third device 4000.
The control unit 3200 can generally control all the operations of the server 3000. For example, the control unit 3200 can control all the operations of the server 3000, executing programs stored in the DB 3300.
In more detail, the 3200 control unit can
136 converting at least one of a text from a first user that is received from the first device 1000 and a text from a second user that is received from the second device 2000 to a voice. The control unit 3200 may determine a pitch of a voice that is converted from the text of the first user, in consideration of user information (eg, a gender and an age) about the first user. In addition, the control unit 3200 can determine a tone of a voice that is converted from the second user's text, taking into account user information (eg, a gender and an age) about the second user.
In addition, the control unit 3200 can transmit the voice that is converted from the text of the first user to at least one of the first device 1000 and the second device 2000 through the communication unit 3100. In addition, the control unit 3200 can transmit the voice that is converted from the text of the second user to at least one of the first device 1000 and the second device 2000 through the communication unit 3100.
In addition, the control unit 3200 can receive a text that is entered by the first user from the first device 1000 through the communication unit 3100 and can transmit the received text to the second device 2000. In addition, the control unit 3200 can receive a text that is entered by the second user
137 from the second device 2000 through the communication unit 3100, and can transmit the received text to the first device 1000.
The control unit 3200 can convert at least one of a first user voice that is received from the first device 1000 and a second user voice that is received from the second device 2000 into text.
The control unit 3200 can transmit a text that is converted from the voice of the first user to at least one of the first device 1000 and the second device 2000 through the communication unit 3100. In addition, the control unit 3200 can transmit a text that is converted from the voice of the second user to at least one of the first device 1000 and the second device 2000 through the communication unit 3100.
The control unit 3200 can receive a voice that is input by the first user from the first device 1000 through the communication unit 3100 and can transmit the received voice to the second device 2000. Also, the control unit 3200 can receive a voice that is input by the second user from the second device 2000 through the communication unit 3100, and can transmit the received voice to the first device 1000.
The 3200 control unit can store a text
138 that indicates a conversation between users making a voice message call. The control unit 3200 can store a part and / or all of the text that indicates the conversation between the users conducting the voice message call. The control unit 3200 can obtain a text that is converted from a voice that the first user enters the first device 1000, a text that is converted from a voice that the second user enters the second device 2000, a text that the first user enters the first device 1000, and a text that the second user enters the second device 2000.
The control unit 3200 can obtain the text indicating the conversation between users of at least one of the first device 1000, the second device 2000 and the server 3000. In addition, the control unit 3200 can store a part and / or all of the text obtained in DB 3300.
The control unit 3200 can store voice data that indicates a conversation between users conducting a voice message call. The control unit 3200 can store a portion and / or all of the voice data indicating the conversation between the users conducting the voice message call. The control unit 3200 can obtain voice data that is converted from a text that the first user enters the first device 1000,
139 voice data that is converted from text that the second user enters the second device 2000, voice data that the first user enters the first device 1000, and voice data that the second user enters the second device 2000.
The control unit 3200 can obtain the voice data indicating the conversation between users of at least one of the first device 1000, the second device 2000 and the server 3000. In addition, the control unit 3200 can store a part and / or the all the voice data obtained in the DB 3300.
The control unit 3200 can generate a call list in connection with a voice message call between users. Additionally, the control unit 3200 can match the stored text and stored voice data with the call list in connection with calling voice messages between users.
The DB 3300 can store data for a voice message call between at least two of the first device 1000, the second device 2000, and the third device 4000.
Voice DB 3310 can store a plurality of pieces of user voice data. The 3310 Voice DB can store voice data that is entered by a user and voice data that is converted from text
140 that the user enters. In addition, the voice DB 3310 can store a plurality of pieces of attribute information about the plurality of pieces of voice data of the users. Attribute information on voice data can be used to distinguish between a voice that is entered by the user and a voice that is converted from text that is entered by the user. However, one or more embodiments of the present invention are not limited thereto.
The text DB 3320 can store user texts. The 3320 Text DB can store a text that is entered by the user and a text that is converted from a voice input by the user. Additionally, the text DB 3320 can store a plurality of pieces of attribute information about user texts. Attribute information about a text can be used to distinguish between a text that is entered by the user and a text that is converted from a voice that is entered by the user. However, one or more embodiments of the present invention are not limited thereto.
The call list DB 3330 can store the call list in connection with the call of voice messages between users. The 3330 call list DB can also store various types of information about the 3330 call list DB. The data stored in the call list DB
141 Call list 3330 can match data stored in voice DB 3310 and data stored in text DB 3320.
The exemplary embodiments may be realized as computer-readable code / instructions on a recording medium, eg, a program module to be executed on computers, that include computer-readable commands. The computer storage medium can include any usable medium that computers can access, volatile and non-volatile medium, and removable and non-removable medium. In addition, the computer storage medium may include a computer storage medium and a communication medium. Computer storage medium includes all volatile and non-volatile media and removable and non-removable media that are technically implemented to store information including computer-readable commands, data structures, program modules, or other data. The communication medium includes computer-readable commands, a data structure, a program module, other data such as modulation-type data signals such as carrier signals, or other transmission mechanism, and includes other information transmission media.
Throughout the description, a term unit indicates a hardware component such as a processor or a
142 circuit, and / or a software component that is executed by a hardware component such as a processor.
It should be understood that the exemplary embodiments described herein are to be considered in a descriptive sense only and not for the purpose of limitation. Descriptions of features or aspects within each modality should typically be considered available for other similar features or aspects in other modalities. For example, the configuration of elements that are singular shapes can be run distributed, and also, the configuration of distributed elements can be combined and then run.
Although exemplary embodiments have been shown and described in particular, those skilled in the art will understand that various changes can be made in form and detail without departing from the spirit and scope of the present invention as defined in the following claims.
It is noted that, in relation to this date, the best method known to the applicant to carry out the present invention is the one that is clear from the present description of the invention.
143
44 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34 Sheet 35 Sheet 36 Sheet 37 Sheet 38 Sheet 39 Sheet 40 Sheet 41 Sheet 42 Sheet 43 Sheet 44
45 members in 9 offices
Priority claims14
| Document | Office | Kind | Date |
|---|---|---|---|
| 1020140062569 | Republic of Korea | – | |
| 20140062569 | Republic of Korea | A | |
| 20140062569 | Republic of Korea | A | |
| 1020150071231 | Republic of Korea | – | |
| 20150071231 | Republic of Korea | A | |
| 20150071231 | Republic of Korea | A | |
| 2015005143 | Republic of Korea | W | |
| 2015005143 | Republic of Korea | W | |
| 1020140062569 | – | – | – |
| 1020150071231 | – | – | – |
| KR20140062569 | – | – | – |
| KR20150071231 | – | – | – |
| PCTKR2015005143 | – | – | – |
| WO2015KR05143 | – | – | – |
Members45
| Document | Office | Kind | |
|---|---|---|---|
| CN105100511A | China | A | |
| EP2947861A1 | European Patent Office (EPO) | A1 | |
| US2015340037A1 | United States of America | A1 | |
| US2015341481A1 | United States of America | A1 | |
| WO2015178715A1 | World Intellectual Property Organization (WIPO) | A1 | |
| KR20150135134A | Republic of Korea | A | |
| AU2015202807A1 | Australia | A1 | |
| US9456074B2 | United States of America | B2 | |
| US2017013106A1 | United States of America | A1 | |
| MX2016015285AThis record | Mexico | A | |
| JP2017517228A | Japan | A | |
| US9736292B2 | United States of America | B2 | |
| US2017331938A1 | United States of America | A1 | |
| US9906641B2 | United States of America | B2 | |
| RU2016150546A | Russian Federation | A | |
| US2018198903A1 | United States of America | A1 | |
| US10075578B2 | United States of America | B2 | |
| EP3393112A1 | European Patent Office (EPO) | A1 | |
| CN108810291A | China | A | |
| RU2016150546A3 | Russian Federation | A3 | |
| AU2015202807B2 | Australia | B2 | |
| EP2947861B1 | European Patent Office (EPO) | B1 | |
| AU2019202418A1 | Australia | A1 | |
| US10284706B2 | United States of America | B2 | |
| EP3496377A1 | European Patent Office (EPO) | A1 | |
| RU2692763C2 | Russian Federation | C2 | |
| US2019245962A1 | United States of America | A1 | |
| MX369624B | Mexico | B | |
| CN105100511B | China | B | |
| CN110875878A | China | A | |
| CN110933238A | China | A | |
| JP2020065293A | Japan | A | |
| JP6688783B2 | Japan | B2 | |
| AU2019202418B2 | Australia | B2 | |
| EP3496377B1 | European Patent Office (EPO) | B1 | |
| EP3393112B1 | European Patent Office (EPO) | B1 | |
| US10917511B2 | United States of America | B2 | |
| KR102225401B1 | Republic of Korea | B1 | |
| EP3793178A1 | European Patent Office (EPO) | A1 | |
| EP3793178A1 | European Patent Office (EPO) | A1 | |
| CN108810291B | China | B | |
| JP6925458B2 | Japan | B2 | |
| CN110875878B | China | B | |
| CN110933238B | China | B | |
| EP3793178B1 | European Patent Office (EPO) | B1 |
Numbers
- Publication
- 2016015285
- Publication, DOCDB
- 2016015285
- Publication, EPODOC
- MX2016015285
- Application
- 2016015285
- Application, DOCDB
- 2016015285
- Application, EPODOC
- MX20160015285
Titles2
- Spanish
- SISTEMA Y METODO PARA PROPORCIONAR SERVICIO DE LLAMADA DE MENSAJES DE VOZ.
- English
- SYSTEM AND METHOD TO PROVIDE VOICE MESSAGE CALL SERVICE.
Classification
- CPC, 20
- H04L51/046
- H04M3/42
- H04M1/72433
- H04L51/066
- H04M3/42382
- H04M3/42391
- H04M7/0042
- H04M2201/40
- H04M2201/60
- H04M2207/18
- H04W4/12
- H04W4/14
- H04W4/18
- H04L51/56
- H04M3/533
- H04W88/02
- G10L15/26
- H04M1/72436
- H04M1/651
- H04M2250/60
- IPC, 7
- H04W4 18
- H04W88 02
- H04M1 72433
- G06F40 00
- G06F40 189
- G06F40 191
- H04M1 72436