System and method for omnichannel user engagement and response

SYSTEM AND METHOD FOR OMNICHANNEL USER ENGAGEMENT AND RESPONSE

Field of Invention

[0002] The present embodiments generally relate to an omnichannel system to fulfill a user request.

Background

[0003] Many companies with products and service offer an automated telephone response system or a website or both.

[0004] A user doing business or intending to do so with the company, could visit the website, potentially create a profile in a secure location and go about doing business. A website could have an internet version and a mobile version tailored to be viewed on a smaller form factor mobile device. In the meantime, to extract the best service and

Date Reçue/Date Received 2022-09-22 ensure seamless experience, many companies offer thin client user mobile applications orapps residing on the mobile device.

[0005] A user could do business either in person (e.g. in a bank), via the Internet, via a mobile app, or via a voice or telephone call.

[0006] In the case of a phone channel, an automated telephone response system is a telephone based communication technology that provides a waiting party with services based upon a waiting party's responses to various prompts. A product of service provider, a queuing party, will typically publish a telephone number and connect the telephone line to a computer system that detects voice and touch tones from an incoming telephone call. The queuing party then creates a series of messages that are accessed by the waiting party. The messages typically give menu choices or information that the waiting party can choose from by either voice response or touching a number on their telephone. Such automated telephone response systems include Interactive Voice Response (IVR) systems, Automatic Call Distributors (ACD), Private Branch Exchanges (PBX), and Voice Over IP (VoIP) systems, to name a few.

[0007] An information system like an automated telephone response system or a website or mobile app can also provide more complex services and give dynamically created messages. There are static and dynamic contents. While, for instance, a financial institution, may have a code for displaying or a script or a pre-recorded messages for speaking their mailing address, or for local branch locations, they will also provide services sud) as account balance information, payment due date, or the ability to make a payment over the telephone or via a web browser or a mobile application. When a user wishes to know their account balance, the automated telephone response system or the website may enquire about the account number or user login information, the user name or other identifying information such as a customer PIN number or other authentication factors such as answer to predetermined questions.

[0008] One disadvantage of service centers is that different options for a user to get to a desired end are often siloed and don’t talk to each other. For example, a customer trying do an online transaction on a website might need to talk to a representative briefly

CA 2989181 2017-12-15 to resolve an issue which is not part of the menu. Moreover, upon switching to that representative the user has to start the whole transaction over (i.e. the context is not maintained).

[0009] Other examples may involve a customer who would like to hang up the phone and continue a transaction on a computer or mobile device. Yet other examples might be handing a call from one device, say a landline phone, to another device, like a mobile phone, or switching to a larger form factor for more clarity and functionality.

[0010] From the foregoing, it will be apparent that there remains a need to improve the user experience with siloed systems or an integrated information delivery system, so that the context of any transaction is maintained while switching between those systems.

Summary

[0011] Embodiments are presented by way of example and include the following:

[0012] One embodiment presents an omnichannel system comprising: an interface configured to receive a request from a user; a web crawler in communication with the interface and configured to fulfill foe request by navigating foe web; a telephone subnet crawler in communication with foe interface and configured to fulfill foe request by navigating an automated telephone response system.

[0013] One embodiment presents an omnichannel system comprising a local interface in communication with a cloud based speech-to-text module and text-to-speech module in a first cloud platform in communication with the local interlace, a cloud based natural language processing module in a second cloud platform in communication with the first cloud platform.

[0014] One embodiment presents an omnichannel system comprising a chatbot, foe chatbot comprising a user interface configured to obtain a user input, a natural language processing (NLP) module configured to obtain the user input and generate a request category, a request indexed database configured to obtain the request category and

CA 2989181 2017-12-15 identify a specific request type, an answer database configured to obtain the specific request and provide an answer to the specific request type and send the answer to the user interface.

[0015] One embodiment presents a method for fulfilling a user request, the method comprising: receiving the user request through a phone channel; transferring the user request to a chat channel configured to fulfill the user request; continuing to navigation on the phone channel to fulfill the user request in parallel.

[0016] in one embodiment a chatbot engages in a conversation with a user who has a request, the chatbot attempts to ask for relevant data from the user, a natural language processing unit then extracts certain attached data or session data from the chat conversation. A decision engine identifies one or more actions to be performed accordingly, A web crawler then navigates a website using the attached data or session data and attempts to perform one or more functions according to the user request. The web crawler then confirms if the request was fulfilled successfully. Upon the confirmation the chatbot informs the user through a message. If the request is not fulfilled, the web crawler passes the attached data or session data to a telephone subnet crawler which navigates a telephone response system to reach a node most relevant to toe user’s request

[0017] In one embodiment, a system comprises a database comprising a number of active accounts of a user, an interface for displaying user activities, a web crawler for performing an action on behalf of the user. The system may further comprise a telephone subnet crawler for performing an action on behalf of a user wherein toe telephone subnet crawler may be configured to use data from toe database, the interface, and the web crawler to navigate a telephone response system to perform an action.

[0018] in one embodiment a system comprises a chatbot, a database comprising metadata, a natural language processing module, and an interface. The chatbot engages in a conversation with a user and a status of an account is presented to the user via the interface only including the metadata. Further the system is configured to

CA 2989181 2017-12-15 receive a request from the user and perform one or more functions on behalf of the user.

[0019] In one embodiment, a system comprises a chatbot comprising a main interface and a plurality of secondary interfaces, each secondary interface configured to access at least one dedicated cloud based database residing in a dedicated cloud platform, thereby allowing a user to select a secondary interface and fulfill a user request through accessing one dedicated cloud based database.

[0020] In one embodiment, a system comprises a chatbot comprising an interface, a natural language processing module, and a knowledge base repository, wherein the knowledge database is based in part on knowledge obtained from a telephone subnet crawler.

[0021] In one embodiment a computerized method comprises engaging a user in a conversation with a chatbot, using natural language processing to identify a relevant request, using a response generator to provide answers to users request from a knowledge base, and identifying an incomplete knowledge area, and using a telephone subnet crawler to navigate a telephone response system to obtain information to alter the incomplete knowledge area.

[0022] In one embodiment, a system comprises a chatbot having a response repository, the response repository containing a plurality of responses to potential questions from a user, an analysis engine to monitor and assign a score to each response based on how well the response helps fulfill the user request, a correction module to check for a low-score response and edit toe response repository accordingly.

[0023] In certain embodiments, toe correction module edits the response repository at least partially based on information gathered by a telephone subnet crawler.

[0024] In one embodiment a system comprises a telephone subnet crawler and a natural language processing module. The telephone subnet crawler uses a combination of DTMF tones and/or synthesized or recorded speech, and/or software commands to navigate a telephone response system. A speech to text module is configured to

CA 2989181 2017-12-15 translate the spoken menu of the telephone response system into text and a natural language processing model is used to interpret the text Accordingly the system reproduces an IVR menu of the automated telephone response system.

[0025] One embodiment presents a network system for operating a chatbot comprising an interface client residing on a user’s device, a local server in geographical proximity of the user’s device, the local server configured to serve at least one of a speech-to-text module, a text-to-speech module, and a static knowledge base, a remote server in network communication with the local server, the remote server configured to update the static knowledge base and the remote server hosting a dynamic knowledge base.

[0026] in one embodiment an omnichannel engagement system comprises a chatbot, a web crawler, and a telephone subnet crawler. As the chatbot engages a user and receives a request, the web crawler attempts to fulfill the request at each step the chatbot reports back the progress made by the telephone subnet crawler to navigate to an automated telephone response menu and the chatbot obtains user data needed for the telephone subnet crawler to fulfill the transaction.

[0027] in one embodiment a method comprises receiving a call from a user and when the user is put on hold, offering the user to interact via chat channel which would include either or both human chat and automated chat If the user opts to interact via chat, and is on a mobile phone, then an SMS or other type of messaging is sent to initiate the chat interaction. The interaction can then happen via SMS or other type of message . protocols. Alternatively the initial SMS or messaging point to a URL that, when clicked, will take the caller to a web-based chat interface, or a 3rd party chat platform like Facebook Messenger. The user could also open an already installed chat application like Facebook Messenger or Whatsapp Messenger and continue the transaction through those chat channel. Otherwise the user could alternatively be prompted to install a specific chat application for this purpose.

[0028] The system keeps the user’s initial call position in the call queue when toe messaging session is in progress and thereby has two processes continuing in parallel while toe user is on hold for an agent via virtual queuing and may also be interacting

CA 2989181 2017-12-15 with the chat agent or chatbot. If the user request is resolved via chat first, the system opts for dropping the user's spot in the virtual call queue. If the user's turn to speak to an agent arrives first, the chatbot notifies the user and optionally abandons the chat. If the chat is unable to fulfill the user request, the system opts for abandoning the chat and maintaining the user’s spot on the virtual call queue.

[0029] In some embodiments the system is further configured to capture and carry over any information gathered in the call (session information) prior to switching to chat, onto the chat session, so the user doesn’t have to provide any information twice. And, ’ conversely, the system is configured to capture and cany over any session information gathered during the chat to the live (voice) agent when one is reached on the phone. The session information could be carried over through session initiation protocol (SIP) headers, passed via web services, or stored in a query database, or key-value pair stores, or other methods.

[0030] One embodiment presents a system for allowing a user to access a human operator (or live agent) in an automated telephone system having a plurality of subnets, wherein different subnets may have different designated human operators. The system may include an interface configured for requesting information from a user; a search module, connected to the interface, configured to receive the information, and to query a database for a subnet within the automated telephone system based on the information request: a telephone subnet crawler connected to the search module and configured to navigate the subnet to reach a human operator pertinent to that subnet in tiie automated telephone system; and a telephone module linked to the telephone subnet crawler and configured to initiate a call to the user (a call-back) and then connect that call to the live voice agent.

[0031]0ne embodiment presents a system for allowing a user to access information in an automated telephone system having a plurality of nodes which may comprise an interface configured to receive a search term or terms from a user to search for information from an automated telephone system; a search module connected to the interface and configured to translate the search term into a desired node in the

CA 2989181 2017-12-15 automated telephone system; and a telephone subnet crawler connected to the search module and the interface, and configured to navigate to the desired node, wherein the interface is further configured to offer the user an option of fetching the information for the user or initiating a telephone call between the user and the desired node.

[0032]In one embodiment a user chats with a chatbot on a device and, in the meantime, is presented a representation of an automated telephone response system where the user could choose a node of the system at any time during the chat with the chatbot. A telephone subnet crawler is configured to navigate to the desired node indicated by the user. The system is configured to allow the user to switch between chatting with the chatbot and reaching a live agent, or to fetch desired information by querying a database at any time.

[0033] In one embodiment a web crawler is launched to obtain web information about an automated telephone response system, the web information is used as partial input of a telephone subnet crawler to navigate through a telephone response system and reach a particular subnet node. Alternatively the telephone subnet crawler is used to extract information from the subnet node and provide it as text or synthesized speech to a user.

[0034] In one embodiment, the interaction ci the user with a chatbot is analyzed by running analytics through natural language processing. Based on the analysis, a particular node of a telephone response system is identified as the most likely node that pertains to a potential user request. A telephone subnet crawler is then used to navigate the telephone response system to reach that particular node. Whenever needed, a telephone module is used to initiate a call between the user and that particular node to either talk to a human operator or to obtain proper information.

[0035] In one embodiment, a system consists of an indexed telephone subnet data repository, a telephone subnet crawler, and a chatbot. The chatbot extracts information from a user using natural language processing and a query engine to query the data repository for telephone subnet nodes corresponding to information extracted using natural language processing. The telephone subnet crawler then navigates a related

CA 2989181 2017-12-15 automated telephone response system accordingly to reach the node. The result of the transaction is then fed back into a learning module.

Brief Description of Drawings

[0036] Embodiments are described by way of example with reference to the accompanying drawings in which:

[0037] FIG. 1-A is a diagram illustrating the structure of an omnichannel response system according to one embodiment.

[0038] FIG. 1-B is a diagram illustrating the structure of an omnichannel response system according to one embodiment.

[0039] FIG. 1-C is a diagram illustrating the structure of an omnichannel response system according to one embodiment.

[0040] FIG. 1-D is a diagram illustrating foe structure of an omnichannel response system according to one embodiment.

[0041] FIG. 1 -E is a diagram illustrating the state management of an omnichannel response system according to one embodiment.

[0042] FIG. 2 is a diagram illustrating a chatbot according to one embodiment

[0043] FIG. 3-A is a diagram illustrating the cloud implementation of an omnichannel system according to one embodiment

[0044] FIG. 3*B is a diagram illustrating foe cloud implementation of an omnichannel system according to one embodiment

[0045] FIG. 3-C is a diagram illustrating the cloud implementation of an omnichannel system according to one embodiment.

CA 2989181 2017-12-15

[0046] FIG. 4 is a diagram illustrating a user interaction process flow according to one embodiment.

[0047] FIG. 5 is a diagram illustrating an omntchannel system process flow according to one embodiment.

[0048] FIG. 6 is a diagram illustrating a knowledge base management system according to one embodiment.

[0040] FIG.7 is a diagram illustrating a process flow of an omnichannel user engagement and response system according to one embodiment.

[0050] FIG. 8 is a diagram illustrating of an embodiment of toe omnichannel system.

Petalled Description of Embodiments

[0051] In the following detailed description, reference is made to toe accompanying drawings that show, by way of illustration, specific exemplary embodiments. These embodiments are described in sufficient detail to enable those skilled in toe art to practice the embodiments. It is to be understood that the various embodiments, although different, are not necessarily mutually exclusive. For example, a particular feature, structure, or characteristic described herein in connection with one embodiment may be implemented within other embodiments without departing from the spirit and scope of toe embodiments. In addition, it is to be understood that the location or arrangement of individual dements within each disclosed embodiment may be modified without departing from toe spirit and scope of toe embodiments. The following detailed description is, therefore, not to be taken in a limiting sense, and toe scopes of the present embodiments are defined only by the claims, appropriately interpreted, along with the full range of equivalents to which the claims are entitled.

[0052] The present disclosure relates to an interactive integrated omnichannel response system from toe perspective of a user and provides embodiments for implementing a quicker, easier and more satisfying experience without placing any

CA 2989181 2017-12-15 additional burden on the businesses or services that employ such integrated omnichannel response systems.

[0053] Fig. 1A is a graphical representation of an omnichannei system according to the present embodiments. The system comprises interface 101, chat channel 111, voice channel 121, and telephone module 140. In operation, interface 101 serves as an interface to engage a user with a request. Interface 101 could utilize either chat channel 111 or voice channel 121 to fulfill the user’s request Chat channel 111 uses information provided to the interface by the user or information already in the system about the user or the nature of the user’s request to resolve the issue and fulfill the user’s request. If chat channel 111 is not successful or if voice channel 121 is deemed more likely to fulfill the request, voice channel 121 secures a virtual spot in the queue or navigates a telephone response system until it gets to an appropriate node which can address the issue and the user’s request Once the virtual spot reaches the front of the queue or once the node is reached, or about to be reached by the telephone subnet crawler, telephone module 140 initiates a callback to the user. When the user answers, she is connected to a live voice agent (in the first case) or an automated telephone response system (in the second case).

[0054] The node in the automated telephone response, which the omnichannel system might need to reach to resolve the user request, might be attended by a live human agent. In this case the user eventually gets to talk to that (voice) agent.

[0055] Fig. 1B is an illustration of an omnichannel system in accordance with some embodiments. This system includes chatbot 100 and independent communication system (ICS) 130. A user might initially engage with either of these two components. In either case toe user could be given toe option of switching from toe chatbot mode to ICS mode or vice versa.

[0056] The omnichannel of figure 1B shows toe structure of toe two communication channels in more detail. Below chatbot 100 there is access module 180 which is used to access third party information. The access might include accessing databases and changing some data in these databases. The databases might be proprietary to third

CA 2989181 2017-12-15 parties. However, these third parties provide facilities to the chat channel of the chatbot To this end, the omnichannel system also includes access repository 181 which is a compilation of the third party facilities and knowledge of how it could access the third party databases.

[0057] On the voice channel, under ICS 130, there is VoiceXML/CCXML module 190. VoiceXML and CCXML are industry standard protocols for setting up IVRs and other voice-based interactive systems. Some companies or called parties use VoiceXML/CCXML to set up their IVR menu. Accordingly, foe ICS can use those protocols to navigate foe voice menu or perform other actions. VoiceXML/CCXML module 190 navigates the IVR menu using VoiceXML/CCXML and reaches a desired node. Another module seen in figure IB is VoiceXML/CCXML module 191 which has a compilation of called parties which use these protocols.

[0058] Referring now to figure 1C, there is another embodiment where there are two components, namely, internet mapping system 111 and IVR mapping module 121 that relate to this embodiment During system ide times or off peak hours, web crawler 110 navigates websites of interest, and in particular, websites of interest in view of potential or recent user requests, and tries to investigate how to potentially resolve an anticipated user request based on foe mapping.

[0059] Likewise, telephone subnet crawler 120 uses idle times or off-peak hours to navigate and map IVR menus of telephone response systems, and in particular, those of potential interest or recent interest to users and tries to investigate how to potentially resolve an anticipated user request based on the mapping. The system occasionally checks to discover if the off-peak IVR is different from the business hours IVR and navigates both IVR menus. When fulfilling a request, the telephone subnet crawler is aware of the differences in the two IVR menus of business hours and off-peak hours and navigates accordingly.

[0060] Referring now to the embodiments shown in figure 1C, internet mapping module 111 generates a database of web pages and websites in relation with potential user requests. Every time a user request is resolved, this module analyses the

CA 2989181 2017-12-15 information about how it was resolved and what web pages and websites were used and in what order and using what data. Any user data or confidential data is encrypted at rest in internet mapping module 111 and web index 112.

[0061] Likewise, IVR mapping module 131 generates a database of telephone response systems and associated or likely user requests that could be resolved by reaching specific nodes in those telephone response system and indexes the information in telephone subnet index 132.

[0062] Internet mapping module 111 and IVR mapping module 131 could be at the backend of the system or cloud modules and are queried by the frontend modules. Likewise, web index 112 and telephone subnet index 132 couid be at the backend of a system or even cloud modules and could be in different cloud platforms.

[0063] Figure 1D is an illustration of an omichannel system with additional components according to some embodiments. At foe top level, there is chatbot 100 and ICS 130. A user could use any connected device to interact with the interface of either component through voice, voice chat, voice commands, text, or a phone call. ICS 130 has a graphical interface that could be used from any device. Initially a user could access the system with fois graphical interface or could access the chatbot interface. Once a request is received from a user through either channel, it is communicated to the other channel and foe two channel exchange data. Alternatively, there could be a common interface that receives text, query, voice command, or voice call and then foe common interface engages either or both channels to fulfill a request accordingly.

[0064] In foe embodiment shown in figure 1D, under chatbot 100 there is third party interface 151. This module provides a number of interfaces which are at least partially provided by third party partner of the chatbot platform. A user could make a transaction by going through these interfaces which might pertain to an airline, a bank, a grocery store, etc.

[0065] The omnichannel system, in these embodiments, these third party interfeces and modules and when a user has a request she could go through the desired interface.

CA 2989181 2017-12-15

The used interface has direct access to third party database 152 through which the user could fulfill her requests like changing a flight, doing an account transaction, ordering items, etc.

[0066] Under ICS 130 in the voice channel, in these embodiments, lies virtual queuing module 171 which maintains a spot in automated telephone response system 172. As explained, both these channels could be active.

[0067] In some embodiments, as shown in figures 1B-1D, the chat channel and the voice channel are in communication. In particular, chatbot 100 and ICS 130 are in communication. When either of the two units receives a request through a common or separate interface from a user, they communicate to each other in order to verify which module is best suited to resolve the user request This could be done by calculating a probability of resolution of the request through either channel based on past or simitar user interactions in part. In addition, in some embodiments, the two systems could act simultaneously or cooperate to resolve the user’s request as will be explained.

[0068] Moreover, at each stage of processing the user request, different components of each subsystem could communicate to each other. For instance, once the request starts being processed, web crawler 110 and telephone subnet crawler 120 could communicate with each other to decide which one is more suitable to fetch certain data related to the user request.

[0069] In some embodiments, the communication between the different components includes exchanging user data that was provided to either chatbot 100 and ICS 130. These data could include any user verification, authentication data, account numbers, PINs, etc.

[0070] In some embodiments, the communication about the state of the transaction between different components and channels of the omnichannel system is done using SCXML.

[0071] SCXML is State Chart XML- and presents a way of laying out a generic state machine execution including steps/tasks in XML. The file can then be read by any

CA 2989181 2017-12-15 module that knows how to read SCXML in order to understand and process certain steps.

[0072) This is used by systems and services in order to coordinate two or more services or sub-systems that are being delivered over different media or channels. For example, there could be a visual dialog via the internet, and a voice dialog via the PSTN. A user might use both channels for clarity or might switch from one channel to the other for convenience or ease or clarification. It is therefore very important to synchronize the state of the transaction between different channels,

[0073] One exemplary embodiment of an SCXML layout in an omnichannel system is shown in figure IE where interaction manager 160 runs SCXML application 161 which supports visual dialog 162 and voice dialog 163.

[0074] User device 168 could communicate via visual dialog 162 through the internet. It could also communicate with media server 163 through CCXM browser 164 and VoiceXML browser 165.

[0075] CCXML browser 164 is in direct communication with SCXML application 161 while VoiceXML browser 165 is connected to SCXML application 161 through voice dialog 163.

[0076] According to fois embodiment, the state of foe transaction of foe user through user device 168 could go through any of foe channels and foe state of foe transaction is still managed.

[0077] In some embodiment, data, especially user data or any other confidential data, first get encrypted when exchanged between any two components of the system. In some embodiments, different levels of encryption are used for different stages of transaction depending on sensitivity and exposure of foe stage of processing and communication.

[0078] Figure 2 is a graphical representation of chatbot 100 according to some embodiments. At a high level, chatbot 100 indudes interface 101, natural language

CA 2989181 2017-12-15 processor 102, third party interface 103, and third party access module 105. There could be a large number of third party interfaces and third party access modules. Alternatively, the access modules might be integrated upon setting some standards and protocols.

[0079] Chatbot 100 engages in a conversation with a user. The conversation could take place using text or voice and could take place on a mobile device or any other connected device like a laptop/computer or tablet.

[0080] When chatbot 100 receives a voice request, it uses a speech-to-text module, which could be residing on the system itself or provided by a third party, to generate text query. It may then use natural language processing to parse the text and create a request query to see if it matches or corresponds to any the services it hosts.

Alternatively, the chatbot could use XML and VoiceXML to prompt a user to enter or utter her request.

[0081] If there is any issue in toe chat channel and chatbot 100 cannot address toe request or cannot understand it, an exception handler makes a decision accordingly. For instance it may decide that additional questions need to be asked of the user: Ultimately, if there is no other handle to understand toe nature of toe request, chatbot 100 might decide to relay the request to toe voice channel and handle it by reaching a live agent at a specific node of toe corresponding automated telephone response system.

[0082] Figure 3A is a graphical representation of how toe omnichannel system user interface is configured to receive voice commands according to some embodiments.

[0083] Local interface 300, which could be used by toe user interface of either chatbot 100 or ICS 130, could receive and then handle a voice command.

[0084] As seen in toe figure, local interface 300, which for example could be implemented on a mobile device, receives a voice command or query and sends toe voice over to cloud module STT 310, a speech-to-text or automatic speech recognition module converting spoken or generated voice to text, which in these embodiments

CA 2989181 2017-12-15 could be a cloud utility offered by a third party. SST 310 returns a text command or query based on the translation of the voice received. Once the command or query is translated to text command or query, it is sent to NLP 320, which could be another third party utility. NLP 200 uses natural language processing to infer or interpret the nature of the query. Once the nature of the query is identified, Al unit 330 could handle the request or direct the right resources to do so. Al unit 330 or part of it, in these embodiments, could be yet another third party utility.

[0085] Al unit 330 could create a (partial) response or a request for clarification from the user. In any of these cases, for a voice session, the output of Al unit 330 is used by TTS 340 to generate speech. TTS 340 could be another third party utility and could be integrated with STT 310.

[0086] TTS 340 then generates speech or uses recorded voice or speech and sends the speech to local interface 300 where it is uttered to the user by the mobile device.

[0087] The embodiment of Figure 3B shows another arrangement where different components of the system are in different cloud platforms with different accessibility speed. The modules STT 310 and TTS 340 are highly accessible cloud components and could be hosted in a cloud platform with high accessibility. Both modules could be third party utilities and integrated with the system.

[0088] Further out in figure 3B, NLP module 320 is shown on a different cloud platform. NLP module 320 could be a third party utility residing on private or public cloud. The access speed to the NLP module 320 need not be as fast as STT 310 and TTS 340 as it resides on a different platform.

[0089] Further out in figure 3B, Al 330 and knowledge base 350 are residing on yet another cloud platform and they may be implemented as third party utilities used by or in the system. The cloud platform hosting these units could be a slower access platform for example as these modules are involved in and perform deep learning.

[0090] In figure 3C, these latter two modules, namely Al 330 and knowledge base 350 are separated into different cloud platforms. Each cloud platform module, further

CA 2989181 2017-12-15 out in the figure, acts as toe backend module for the modules closer to the local interface in figure 3C.

[0091] The user request is processed by NLP module 320 and it is interpreted or translated into query language, understandable by Al 330. This module looks to provide an answer or feedback at a certain stage of a conversation or engagement with a user. Al 330 may use deep learning to train for ever more effective resolution of user requests.

[0092] Knowledge base 350 is in communication with Al 330. The accumulated knowledge learned by Al 330 is stored in this database for future or anticipated user request resolution. This is in addition to any initial or periodic knowledge or data about user request resolution already stored in knowledge base 350, for instance by a programmer.

[0093] Figure 4 is a graphical representation of an exemplary transaction between a user and an omnichannel system according to some embodiments. At step 410 toe chatbot is engaged in a voice or text conversation with a user upon an inquiry or request or based on anticipation. At step 411 the user expresses their wish to change an international flight. The chatbot then may use NLP to decipher user's request and uses Al to address toe request. Alternatively this interaction could be based on XML or VoiceXML menu presentation and receiving user input and categorizing the request

[0094] At step 412, the chatbot gathers any user data related to this request based on user profile or based on previous interactions or access to applications related to the international flight that reside on toe same device on which toe chat is taking place. Alternatively, toe chatbot uses cloud residing data or transaction records that might relate to this request. If needed, the chatbot could prompt toe user tor more information.

[0095] At step 413 toe chatbot uses the request information as well as user data and attempts to access a third-party database to fulfill the request. As a specific example, the chatbot might attempt to access the airline database or other travel authority database to change a flight according to a user request. Once the chatbot manages to

CA 2989181 2017-12-15 perform this task, the confirmation is returned and the chatbot uses text or uses a TTS (text-to-speech) module and generates voice to inform the user through text or voice respectively.

[0096] At step 420 the system attempts to confirm whether the user’s request has been fulfilled. This may happen after more than one attempt to clarify what the nature of the user’s request is and gathering user data at multiple steps and crawling foe website or querying/modifying a flight database.

[0097] If foe request is fulfilled and foe user is satisfied, at step 430, foe chatbot informs foe user that the request has been fulfilled and provides new flight information at step 431 through foe interface.

[0098] If the request is not fulfilled, at step 440, the ICS system takes over fulfilling the request and obtains all the user data and other data related to the request from foe chat channel. This is part of the handover of the request from a web-based system to a telephone based system, or from chat channel to the voice channel.

[0099] At fols point, or at any other instance deemed reasonable, the system might decide that foe user does not need to stay on the line or be engaged in a that conversation. The user therefore could go about usual business or leisure and is later informed of foe state of foe request fulfillment as explained below.

[00100] At step 441, foe ICS queries an indexed repository of automated telephone response system data repository to find out about relevant telephone numbers and their IVR structure as they relate to the best way of fulfilling the user request. Alternatively, and specially if the number is not in the repository, the ICS could attempt to use VoiceXML and CCXML to reach a relevant automated telephone response system or a node in foe automated telephone response system.

[00101] At step 442, a node in an automated telephone response system is Identified as the most relevant node to address the request. The node might be represented by a live agent in an automated telephone response system that is most relevant to the

CA 2989181 2017-12-15 particular request. For instance, in this example, the node might be an agent that handles changing international flights or reservations of an airline.

[00102] Once the node is confirmed, at step 443 the ICS attempt to get to the node. In some embodiments this is done using VoiceXML/CCXML navigation of the automated telephone response system.

[00103] In some embodiments, the telephone subnet crawler navigates the telephone response system to reach a node in foe automated telephone response system relevant to the user request. The telephone subnet crawler utilizes foe obtained user data and user information, such as flight confirmation codes and flight information to navigate to the node and to facilitate potential handover to a relevant live agent. In doing so, the telephone subnet crawler may use any combination of DTMF, recorded or generated voice, and software commands. The telephone subnet crawler could use a mapped or previously obtained IVR menu of the automated telephone response system for more efficient navigation.

[00104] Once a live agent representing foe node is reached or is about to be reached, at step 448, the ICS confirms that the relevant node has been reached. This could take place once the navigation is close to reaching foe node, for instance, one step away from foe node as it does know toe IVR map.

[00105] At step 450 the chatbot, or any relevant interface, informs the user that a connection is made or is about to be made. For instance, the chatbot or interface might first pop up a message or text or a notification informing the user that reaching a relevant node operator related the change international flight or reservation at the airline is imminent. The user then could prepare to be ready to talk to an operator.

[00106] At step 451 foe telephone module places a call to the user (a call-back) and connects foe user to foe agent operator relevant to the identified node. Finally, at step 452, toe user could talk to the relevant operator to resolve foe request.

CA 2989181 2017-12-15

[00107] Referring now to figure 5, this figure is a graphical representation of some of the steps the omnichannel system makes to train Itself for better handling of a request according to some embodiments.

[00108] At step 510 the omnichannel system receives a request from the user. The request could be in the form of text, voice command or voice call. If the request is with voice command or query cr through a voice call, the STT module translates the voice into text

[00109] At step 511 the NLP module uses natural language processing and translates the text into a command or a query language understandable to the system.

Alternatively this could be done through menu presentation and selection using XML or VoiceXML.

[00110] At step 512 the system provides options to the user. This could be done by either XML or VoiceXML menu presentation or could be done by a teaming or (narrow) artificial intelligence AI/NAI module through deep learning.

[00111] At step 515, the system provides a third party interface that might be residing on the chatbot system or might be autonomous. The system then interacts with the user and tries to fulfill the user request by accessing a relevant third party database at step 518.

[00112] Once a response or a resolution has been reached, the system looks for feedback and analyses and reevaluates its course of action and the best action or channel or interface. This learning process goes on and the omnichannel system gets more and more efficient and faster at handling various requests or situation.

[00113] To this end, at step 520 the system receives feedback from the user. The feedback could be anything from a one bit yes or no question to a scoring system or a sentence or two getting the feedback in voice or text and potentially using the NLP module at step 521 to extract feedback information.

CA 2989181 2017-12-15

[00114] The system then assesses the response at step 522, and calculates a score for foe response or its course of action at step 523 based on the feedback. This process could be applied to each step in a course of action that the system takes.

[00115] Each time a question or request is categorized, a response or course of action is chosen at step 530 based on what foe system has learned. To handle further requests, the system then takes a course of action according to what it has learned at step 531. This turns into an iterative learning process.

[00116] If, upon feedback and valuation, a course of action turns out to be satisfactory, at step 540 foe system opts to keep that response or course of action forthat particular category or expand it to similar categories. If the course of action is not satisfactory, at step 550, the system opts to modify the course of action for the future.

[00117] Likewise, as shown in figure 6, the voice channel learns from interactions with a user. At step 610, the system starts with a knowledge base. At step 611 the system receives feedback. This could be during the course of a transaction or after a transaction.

[00118] At step 612, based on the feedback, the system navigates relevant automated telephone response systems and obtains information from the subnets of various automated response systems, and maps their IVR menu and gathers other information at step 613.

[00119] At step 614, the system compares information gathered by the navigation with the knowledge base and modifies the knowledge base accordingly at step 615.

[00120] Referring now to figure 7, the figure is a graphical representation of an interaction between a user having a request and an omnichannel system according to some embodiments.

[00121] According to these embodiments, at step 710, a user places a voice call with a request to an automated telephone response system. The voice call might be made on

CA 2989181 2017-12-15 a landline telephone, mobile device, or a VoIP phone. The user is then directed to the omnichannel system.

[00122] At step 711, the user may provide data. The data could be automatically fetched from user profile or from transaction history. Alternatively, the data could be provided by the user on occasions that new or alternative data is required.

[00123] At step 712 the system provides the user the option to use the chat channel instead of, or in addition to, the voice channel and tries to resolve or fulfill foe request via the chat channel. At step 720 foe user responds yes or no to taking foe chat option.

[00124] If the user rejects using a chat channel, at step 730, the independent communication system (ICS) holds a virtual spot for the user on a telephone response system that is relevant to the user request. The ICS could be doud based system and independent of the called party automated telephone response system. For example, if the user is trying to reach an agent in a bank, foe ICS holds a spot for foe user in foe telephone response queue and foe user could go about doing usual business until the agent is reached by foe ICS. In some embodiments, foe ICS uses a telephone subnet crawler to navigate the IVR menu of the automated telephone response system to a specific node of the IVR system. The telephone subnet crawler uses the information from the telephone subnet index and navigates foe IVR menu. Alternatively, foe ICS may use VoiceXML/CCXML to reach a relevant node.

[00125] If fois case happens, at step 770. foe ICS navigates to a node relevant to foe request. This may involve using any of DTMF tones, generated or recorded speech, or software and query commands to navigate foe IVR menu of a called party. Alternatively it may invdve generating VoiceXML prompts to navigate a menu.

[00126] If at step 720, foe user accepts to take foe chat option, branching to step 740, the chatbot starts engaging in a conversation with the user to fulfill foe user request.

[00127] At step 741, foe chatbot collects session data relevant to fulfilling foe transaction. The session data is any data relevant to the user request that has been already collected by foe ICS or is otherwise available from foe user profile. The

CA 2989181 2017-12-15 omnichannel system therefore does not ask for data from a user more than once and even that happens only when needed. Otherwise the user profile data and transaction history is used to obtain these data by the omnichannel system.

[00128] Once the required data is gathered, at step 742, the chatbot attempts to fulfil the request. This might involve, for the example, changing an international flight, getting all the flight confirmation data and user authentication data and accessing airlines database and/or other third party fulfillment websites database.

[00129] One advantage of the omnichannel system is foe fact that, while foe user is engaged by foe chat channel which is trying to fulfill foe request, the voice channel is still running in parallel as an alternative means for fulfilling the request. Depending on which channel is successful or faster, one of several cases might happen as explained below.

[00130] At step 750, one of a few cases might happen. In the first case, at step 760, foe web crawler is able to resolve issues and fulfill the request The chatbot, accordingly, informs the user of foe fulfillment and prorides, in this example, new flight information. Once the user is satisfied with the fulfillment of the request, at step 761, foe ICS drops the virtual spot in the queue of foe automated telephone response system, and the telephone subnet crawler drops its navigation through the IVR menu.

[00131] In the second case, at step 790, if the chatbot gets stock and cannot fulfill the request. In this case, the chat channel stops and instead the ICS, as seen at step 770, continues navigating to the node identified as the most relevant to foe request, potentially with a human agent at that node.

[00132] it should be noted that, in some embodiments, the ICS is always navigating the automated telephone response system regardless of the stage of the progress in foe chat channel. This is in anticipation of the chat channel progress getting stick, as it is illustrated in step 790. Alternatively foe chat channel may be taking a long time addressing foe request.

CA 2989181 2017-12-15

[00133] In some embodiments, the user is offered to take the chat channel or uses the chat channel to begin with, however, the chatbot or chat channel gets stuck, and cannot fulfill toe request. The system then uses the voice channel and, since toe user has spent considerable time on the chat channel, she is moved in front of the queue of the corresponding automated telephone response system. In these embodiments there is no need to run both channels in parallel all the time. However, the voice channel is aware of toe time toe user has spent on the chant channel through system state control, and grants the user a virtual spot accordingly such that toe user doesn't feel she or he has lost all toe time on toe chat channel. One advantage of these embodiments is that the system does not use voice channel resources unless it needs to do so. In addition, the user doesn't have to wait as long. The live agent benefits from these embodiments by saving time and knowing the nature of the user request and toe issues beforehand as well.

[00134] In the third case, the ICS navigates to the node and reaches toe node while the chat channel is still tying to address the request, in this case, the user is informed that a relevant node has been reached. This might potentially be with a live agent, which in this case, might be toe international flights reservation of an airline.

[00135] At any of these cases leading to the voice channel resolving the request faster, at step 780, the ICS verifies the node. The ICS then might opt to confirm that indeed a live agent at the node is readied.

[00136] According to some embodiments, the ICS verifies that it has indeed reached a live agent or human operator by registering transition audio queues. During navigation, the ICS listens for audio queues, and navigates to a desired node. The audio queues indicating reaching a live agent are called transition audio queues.

[00137] The transition audio queues may be recognized by the ICS when toe automated telephone response system hands over toe call to a live agent of the called party. The live agent could ben distinguished by listing to a distinguished sound or click while navigating the IVR menu.

CA 2989181 2017-12-15

[00138] Alternatively or in addition, the ICS could confirm that an agent has indeed been reached by posing a challenge question. Atypical challenge question would be something like: “are you a real person?” and listening for “yes or alternative positive uttering or “are you a computer? and listening for “no” or alternative negative uttering. Yet another way to ascertain of the existence of an agent is to prompt the agent for a DTMF tone, for instance, “please press 1 if you are a human agent”.

[00139] Another way to distinguish transition audio queues is listening for background noise, which could be different for the automated stage and the live agent stage.

[00140] Immediately after reaching a node is confirmed, at step 781, the telephone module of the omnichannel system gives the user a call back and connects the user to the live agent.

[00141] It should be noted that, in some examples, the desired node might not be a live agent but a place in the IVR menu where the user could directly fulfill his or her request. For instance, a voice recording in the automated telephone response system might confirm that the flight has been changed and provide new information using recorded or generated speech.

[00142] According to the present embodiments, a user could pursue fulfilling his or her request on one channel, while keeping his or her spot on another channel. Alternatively, a user could start interacting with the omnichannel system of present embodiments on one channel, and enable other channels to run in parallel in order to provide the fastest route to securing fulfillment of his or her request.

[00143] Referring now to the embodiment illustrated in figure 8, the omnichannel system operates Internet channel and telephone channel through the interface layer 810.

[00144] Interface layer 810 includes phone interface 812 with dual-tone-multifrequency (DTMF) dialing and sound capability, a web graphical user interface (GUI) 814, and service application programming interfaces (APIs) 816.

CA 2989181 2017-12-15

[00145] Connected to the interface layer 810 is a set of high-level modules 820 that include telephone subnet crawler/indexer software 822, search/browse module 824, statistics module 826 and translator module 828.

[00146] Underlying the high-level modules 820 is a set of low-level modules 830, including text-to-speech/speech-to-text (TTS/STT) module 832, voice-over-internet APIs 834, natural language processing/pattem analyzer 836, and speech synthesizer 838.

Finally, omnichannel system 800 has data storage 840.

[00147] There are variations and modifications that will be readily apparent to those skilled in the art in light of the above teachings. It is therefore to be understood that, within the scope of the claims hereto attached and supported by this specification, the embodiments may be practiced other than as specifically described.