Methods and apparatus for performing an internet search

Method and device for performing internet search

The technology described in this article is generally about the field of sending search queries via the Internet.

The Internet is one of the global interconnected computer network systems that store all kinds of information. The World Wide Web (WWW) is an information sharing model built on the Internet, in which a system of interconnected hyperdocuments uses a specific protocol (that is, the hypertext transfer protocol and its variants) for access.

Because of the large amount of information available through the WWW and the Internet, and because the available information is distributed across a large number of independently owned and operated networks and servers, locating the desired content on the WWW and the Internet presents a challenge.

A search engine has been developed to help users locate the content they want on the Internet. A search engine is a computer program that receives from a user (for example, in the form of a keyword set) a search query indicating what the user wants, and returns the search engine decision and the user's search query Related information and/or hyperlinks to information.

Search engines usually use a computer program called webcrawler (which browses the WWW in an automated way (for example, following every hyperlink it finds in every web page it browses)). Work on a large number of WWW pages and/or other content. Analyze the retrieved webpages and/or content and store information about the webpages or content in an index. When a user sends a search query to a search engine, the search engine uses the index to identify the page and/or content that it determines best matches the users search query and returns the page and/or content with the best match One result list. Frequently, this list is in the form of one or more web pages (which include a collection of web pages and/or content hyperlinks that are determined to best match the user's query).

There are at least two general types of search engines that can be accessed via the Internet: general search engines and site-specific search engines. As used in this article, the term "universal search engine" is defined to mean providing web pages and/or content (or hyperlinks to web pages and/or content) owned on at least two different and independent websites or domains. One of the search results of the search engine. General search engines attempt to index search results from content distributed across the Internet and provide these search results. Examples of general search engines include Google operated by Google, Inc. in Mountain View, California^TM; Yahoo! operated by Yahoo!, Inc. in Sunnyvale, California; and Bing operated by Microsoft Corp. in Redmond, Washington^TM。

As used in this article, the term "site-specific search engine" is defined to mean providing a search that includes webpages and/or content (or hyperlinks to webpages and/or content) owned only on one website or domain One of the results of a search engine. Website operators frequently use site-specific search engines to allow users to find specific web pages or content on their websites. For example, the website of an online retailer (or "e-retailer") may include a site-specific search engine that facilitates a user to locate products sold by the retailer.

An embodiment relates to a method for performing a content search via the Internet at least partially implemented by at least one hardware computer processor. The method includes: receiving input from a user; and in response to receiving the input, sending at least one The search query is sent to a plurality of search engines, wherein the plurality of search engines includes at least one general search engine and at least one site-specific search engine, and the at least one search query is based at least in part on the input content. Another embodiment relates to at least one computer readable medium encoded with instructions that, when executed, perform the above-mentioned method.

A further embodiment relates to a system that includes: at least one tangible memory, the processor of which stores executable instructions to perform a content search via the Internet; and at least one hardware computer processor coupled to the At least one tangible memory, the at least one hardware computer processor executes the processor-executable instructions to: receive input from a user; and in response to receiving the input, send at least one search query to a plurality of search engines, The plurality of search engines include at least one general search engine and at least one site-specific search engine, and the at least one search query is based at least in part on the input content.

Another embodiment relates to a method for performing a content search via the Internet at least partially implemented by at least one hardware computer processor. The method includes: sending at least one search query to a plurality of search engines; At least one search query is sent to the plurality of search engines, and search results are received from at least some of the plurality of search engines; and the search results are displayed on a display of a display device, wherein the displayed search results are classified into A plurality of sets, each set is associated with a specific search engine. A further embodiment relates to at least one computer readable medium encoded with instructions that, when executed, perform the above-mentioned method.

Another embodiment relates to a system that includes: at least one tangible memory, the processor of which stores executable instructions to perform a content search via the Internet; and at least one hardware computer processor coupled to the At least one tangible memory, the at least one hardware computer processor executes the processor executable instructions to: send at least one search query to a plurality of search engines; in response to sending the at least one search query to the plurality of searches Engine, receiving search results from at least some of the plurality of search engines; and displaying the search results on a display of a display device, wherein the displayed search results are classified into a plurality of sets, and each set is associated with a Associated with a specific search engine.

A further embodiment relates to a method for performing a content search on the Internet at least partially implemented by a hardware computer processor, the method comprising: sending at least one search query to a plurality of search engines, the plurality of search engines Including at least one site-specific search engine; in response to sending the at least one search query to the plurality of search engines, receiving search results from at least some of the plurality of search engines, wherein the search results are included from the at least one site At least one webpage returned by a specific search engine; and displaying the at least one webpage returned from the at least one site-specific search engine on a display of a display device. Another embodiment relates to at least one computer readable medium encoded with instructions that, when executed, perform the above-mentioned method.

A further embodiment relates to a system that includes: at least one tangible memory, the processor of which stores executable instructions to perform a content search via the Internet; and at least one hardware computer processor coupled to the At least one tangible memory, the at least one hardware computer processor executes the processor executable instructions to: send at least one search query to a plurality of search engines, the plurality of search engines including at least one site-specific search engine; In response to sending the at least one search query to the plurality of search engines, receiving search results from at least some of the plurality of search engines, wherein the search results include at least one webpage returned from the at least one site-specific search engine And making the at least one webpage returned from the at least one site-specific search engine displayed on a display of a display device.

The inventors have recognized that for a given user to generate a search query, the user may be interested in different types of information about the search query. For example, for the search query "Miles Davis", users may be interested in: obtaining biographical information about Miles Davis; listening to music samples of Miles Davis or purchasing music by Miles Davis; and/or related to those interested in Miles Davis Others engage in social networking. Historically, in order to obtain these three different types of information, a user entered the search string "Miles Davis" into three different search engines. For example, the user inputs this search string into a search engine of an encyclopedia website to obtain biographical information; inputs this search string into a search engine of a website that sells music to listen to or purchase music; and The search string is entered into a social networking site to connect with other people. The inventors have recognized that this procedure is usually time-consuming and laborious, because the user has to browse multiple different websites (and potentially manage multiple different browser windows or tabs) and repeatedly enter the same search string.

The inventors have also recognized that because different search engines index web pages and/or content in different ways; index content in different fields; and/or use different algorithms to determine which web pages and/or content best match A specific search query, so different search engines can provide different (although possibly overlapping) search result sets in response to the same search query. This concept is illustrated by Fan's diagram in Figure 1. Each set depicted in the Fan diagram in Figure 1 represents one of the four hypothetical search engines (ie, search engine 1, search engine 2, search engine 3, and search engine 4) responding to the keyword "Miles Davis "One of the search results provided by a user who provided a search query. In FIG. 1, the set 101 includes the results provided by the search engine 1 and includes the search results "Q", "R", "S" and "T". The set 103 includes the results provided by the search engine 2 and includes the results "R", "S", and "Y". The set 105 includes the results provided by the search engine 3 and includes the results "S", "T", and "X". The set 107 includes the results provided by the search engine 4 and includes the result "Z". As shown in Figure 1, some search results are contained in only one of the collections and are therefore returned from the only one of the four search engines, while other search results are contained in multiple collections of the collections And is therefore returned from two or more search engines.

When a search engine returns a set of search results in response to a search query, the search engine usually returns the results in an ordered list. The list may be sorted by relevance, may be sorted based on the money paid for higher positions in the search results, and/or may be sorted based on other criteria. For example, a webpage or content that has been determined by a search engine as the most relevant is at the top of the result list, and a webpage or content that has been determined by the search engine as less relevant is at the lower end of the result list. As another example, a specific electronic device manufacturer can pay an electronic device e-retailer to enter a search query containing the word "TV" into the site-specific search engine of the e-retailers website Make its TV set higher in the list.

The inventor has understood that because different search engines use different algorithms to determine the order in which web pages and/or content are listed in the search results, even when two different search engines respond to a specific search query (for example, , "Miles Davis") and the same webpage or piece of content is included in its search results, the webpage or piece of content may be at the top of the result list provided by the first of the two search engines Or nearby (for example, because the first search engine has determined that the page or piece of content is particularly relevant in the content area it evaluates), but may be in the list of results provided by the second of the two search engines Compared to the lower end (for example, because the second of the two search engines has determined that the webpage or piece of content is less relevant in the content area it evaluates). Similarly, some results listed near the top of the list of search results from the second search engine may be listed at the lower end of the list of results provided by the first engine. The inventors have recognized that users are more likely to notice and access results near the top of the list. Therefore, by using only one of the two search engines, a user may not notice or access results that may be highly relevant to the user.

Therefore, the inventors have recognized that sending a user-specified search query to only a single search engine increases some potentially relevant web pages and/or content pieces that are not included in the search results or are listed in the returned search results list The probability that it is low enough for users to ignore it. In addition, the inventors have recognized that sending this search query to only a single search engine can limit the returned content to the type of content searched and/or indexed by the search engine, and the user may be Interested in many different types of content about this search query.

Some web-based software programs (called metasearch engines or search engine aggregators) have been developed, which receive a user-specified search query; send the search query to multiple search engines; and so on Each of the search engines receives the results; removes duplicates; aggregates these search results into a single list; and displays this aggregated search result list to the user. An example of this meta search engine is Dogpile operated by Infospace, Inc. in Bellevue, Washington.^TM. However, the inventors have recognized that these meta search engines have many shortcomings.

First, the search engines queried by these meta search engines only include general search engines, and do not include any site-specific search engines.

Second, because the meta search engine responds to a user-specified search query and displays to the user a single list of aggregated search results from many different search engines, the user does not have the ability to see which search engine provides which search Results; see how each of the multiple search engines thinks a particular search result is relevant; or view the complete set of results returned from any particular search engine.

Third, the meta search engine runs on one or more servers (which are received from a browser or other software application running on the users client device and sends a user-specified search query), and the The search query is sent from the server(s) to a plurality of search engines to be queried. Because a meta search engine can receive search queries from a large number of users and provide aggregated search results to these users, the server(s) on which the meta search engine can be operated can perform many searches The query is regularly sent to the search engine whose results are summarized by the post-search engine. One of the results is that a search engine can receive hundreds of thousands or even millions of search queries from the same server or IP address every day, and this behavior can be regarded as a single user sending a huge number of search queries To the search engine. Many search engine operators regard this behavior as an abuse of search engine services and/or potential denial of service attacks and will take actions to prevent search queries from sending such a large number of queries from one of the IP addresses to their search engines. For this reason, a meta search engine operator usually has to reach a contractual agreement with the search engine operator whose search engine results are aggregated by the meta search engine to allow the meta search engine to send a large number of search queries to these search engines.

Fourth, these meta search engines do not provide the user with the ability to control which search engine his or her search query is provided to. The fact is that the existing meta search engine has a fixed set of search engines provided for each search query. Therefore, the user does not have control over which search engine results are provided in response to a search query, and the post-search engine does not execute which search engine to query based on which user sent the search query or based on the content of the users search query Any customization.

Some embodiments described below relate to technologies related to sending a user-specified search query to multiple search engines and/or displaying the results of such queries to the user. Although some embodiments discussed below solve all the above-mentioned deficiencies of the existing meta search engine, not every embodiment solves all the above-mentioned deficiencies of the meta search engine, and some embodiments do not solve any of these deficiencies. Therefore, it should be understood that the present invention is not limited to the embodiment of the owner or any one of the above-mentioned defects of the meta-search engine.

FIG. 2 is a flowchart of a diagrammatic explanatory process 200. In some embodiments, the process 200 can be executed by executing an application program on a user's client device to send a search query to multiple search engines And to show users the search results from each of these search engines. The client device on which the application program is executed to execute the program 200 may be any type of computing device having hardware capable of executing a software computer program. Examples of the types of client devices that can be used include a laptop or desktop personal computer, a personal assistant (PDA), a mobile phone, a server computer, and/or various other types of computing devices. In a specific implementation described herein, the technology described below can be implemented on a mobile phone (eg, a smart phone) and can be combined with a voice recognition capability, so that users can send search queries by voice , But all aspects of the present invention are not limited to use on mobile phones or use with a voice recognition interface.

As can be understood from the above description, the procedure 200 allows a user to launch an application on the client device, (for example, by voice or by text) to enter a search query and observe the information provided by multiple search engines. The search result of the search query.

A non-limiting example of an environment in which the executable program 200 is executed is in the network computer environment 300 shown in FIG. 3. In FIG. 3, a user 301 accesses the Internet 305 via a client device 303. The search engines 307a, 307b, 307c, ..., and 307n are executed on servers that can be accessed via the Internet 305. The process 200 starts with action 201, in which the application program running on the client device 303 receives a user-specified search query. As discussed in more detail below, queries can be received in any of a variety of possible ways via any of a variety of possible client device user interfaces. Then, the process continues with action 203, where the application program determines which of the search engines 307 to query in response to receiving the user-specified query. As discussed in detail below, the set of search engines to be queried can be dynamically determined and user-configurable. However, the aspect of the present invention is not limited in this respect, because in some embodiments, the search to be queried The collection of engines can be static and not configurable.

Then, the process proceeds to act 205, where the application sends the search query generated based on the user-specified search query received in act 201 to the search engine 307 identified in act 203 via the Internet 305. In some embodiments, these search engines may include general search engines and/or site-specific search engines, but the present invention is not limited in this respect, because in some embodiments, only site-specific searches can be queried Engine or general search engine. Then, the process continues with action 207, in which the application receives the search results from the search engine queried in action 205 via the Internet 305. Then, the program proceeds to action 209, in which the search result is displayed to the user via a display on the client device 303.

It should be appreciated that program 200 depicts an illustrative sequence of one of executable actions 201, 203, 205, 207, and 209. Various other orders are possible, and these actions may be performed in different orders in various different embodiments. In addition, various possible implementations of the various actions of the program 200 are feasible and are discussed in more detail below.

<b>I. Receive a search query</b>

As discussed above, in action 201 of process 200, the application receives user input specifying a search query. This user input can be received in any of a variety of ways and in any of a variety of formats. For example, in some embodiments, user input may be received as text and may be received via an input device integrated into or coupled to the client device. Examples of these input devices include a keyboard, a touch screen, a mouse, and/or any other input devices through which the user can provide text input to a computing device.

In some embodiments, the user input can be received as audio via a microphone integrated into the client device or coupled to the client device. For example, a user can speak a search query into the microphone and the application can receive the user's voice input as audio data. Automatic voice recognition can be performed on the audio data to obtain a recognition result (for example, in text form) that can be used as a basis for a query search engine. In some embodiments, one or more of the search engines to be queried may have a voice interface. In these embodiments, audio can be provided to these search engines instead of, for example, providing a text search query based on automatic speech recognition of the audio. The automatic speech recognition technology is well known, and any one of a variety of automatic speech recognition technologies can be used to obtain one of the recognition results of the audio data.

In some embodiments, the client device can provide many different options, and the user can choose from these options to provide a search query. For example, in some embodiments, the client device may allow the user to use voice format (for example, by speaking a query into a microphone) or in text format (for example, by typing on a keyboard or a touch screen). Query) Enter the query.

In an embodiment in which automatic voice recognition of voice input is performed, the automatic voice recognition of user-supplied audio data can be accomplished in any of a variety of ways. For example, as shown in FIG. 4, in some embodiments, the client device can execute an automatic voice recognizer, which is a software program that performs automatic voice recognition on audio data, and the application program The automatic voice recognizer can be used to obtain a recognition result of audio data. In FIG. 4, the client device 401 executes an application 403 (which executes the program 200) and an automatic voice recognizer 405. When the application 403 receives the audio data 407 provided by the user, the application 403 can pass the audio data to the automatic voice recognizer 405. The automatic speech recognizer 405 can perform automatic speech recognition on the audio data to obtain a recognition result 409 and can send the recognition result 409 back to the application 403 for use in the query of the formatted voice engine.

The inventors have recognized that some client devices may not have sufficient computing resources to execute an automatic voice recognizer that operates at a desired accuracy level and/or efficiency level. That is, for example, a client device may not have enough memory to store sound models, language models, grammars, and/or other components of the speech recognizer and/or may not be powerful enough to perform at a desired efficiency level A processor for automatic voice recognition. This can occur in any number of scenarios, especially when the client device is a handheld device (such as a PDA or mobile phone).

Therefore, in some embodiments, it is possible to perform automatic voice on audio data supplied from a user by a computer outside the client device that executes the application (which receives user input and manages searches based on this input) Identify. For example, as shown in FIG. 5, the client device 501 executes an application program 503, and the application program 503 receives a voice input provided by a user in the form of audio data. The automatic voice recognition of audio data can be performed by an automatic voice recognizer 507 running on a server 505. Therefore, as shown in FIG. 5, in some embodiments, the application can send the user-supplied audio data 509 to the automatic voice recognizer 507 running on the server 505. The audio data 509 can be sent to the automatic voice recognizer in any of a variety of possible ways. For example, in some embodiments, a microphone may be used to capture user-supplied audio and the captured microphone audio may be converted from analog to digital to generate digital audio data. This digital audio data can be supplied to the automatic voice recognizer 507. In some embodiments, some processing of the digital audio data can be performed before sending the data to the automatic voice recognizer. For example, digital data can be compressed or some preprocessing can be performed to retrieve audio parameters useful in automatic speech recognition. The captured parameters can be transmitted to the automatic voice recognizer 507 together with the audio data. In embodiments using compression, any of a variety of possible compression algorithms can be used. For example, in some embodiments, speex can be used^TMencode decode.

Therefore, it should be understood that, as used herein, the term "audio data" can refer to an analog audio signal or a digital representation or a compressed digital representation of an analog audio signal.

The automatic speech recognizer 507 can perform automatic speech recognition on the audio data 509 and return a response 511 (which includes the result from the audio data 509) to the application 503. The results can be in any suitable form. As a non-limiting example, the results may include a text recognition result obtained by the automatic speech recognizer 507 from performing automatic speech recognition on the audio data.

In the example of FIG. 5, the client device 501 is depicted as directly communicating with the server 505. It should be understood that this depiction is only provided to facilitate the understanding of the type of information sent between the client device 501 and the server 505, and the client device 501 can be used in any of a variety of ways and via any appropriate communication medium. (For example, via the Internet) to communicate with the server 505.

In the example of FIG. 4, automatic voice recognition is performed on the audio data received at the client device on a client device, and in the example of FIG. 5, the audio data is provided from the client device to a server And the server performs automatic voice recognition on the server. However, in some embodiments, automatic voice recognition may be performed partly by the client device and partly by the server. For example, part of an ASR can be executed on the client and part of the server can be executed. This is done, for example, for the client device to execute a part of the voice recognition process, but leave a computationally intensive part for the server.

In some embodiments, the server 505 can provide automatic voice recognition services for multiple client devices. FIG. 6 shows a computing environment in which each of a plurality of users 601a, 601b, ... 601n operates a client device 603a, 603b, ..., 603n. Each client device 603 can execute an application program that executes the procedure 200 of FIG. 2 to send search queries to search engines 609a, 609b, 609c, ..., 609n and so on via the Internet 607 The respective users display the results of these search queries. The server 605 can provide an automatic voice recognition service.

Therefore, in an embodiment in which the client device 603 receives user-supplied search queries in the form of audio data, the client device 603 can send the audio data to the server 605 to perform automatic voice recognition on the audio data, and can According to this response, an identification result or other information based on the identification result is received.

In the example discussed above, the server 605 provides an automatic voice recognition service for the client device 603. In some alternative embodiments, the server 605 may provide additional functionality to facilitate the sending of search queries from a client device 603 to one or more search engines 609. Some examples of this additional functionality are described in detail below. For simplicity, in the example of FIG. 6, only one server 605 is depicted. However, it should be understood that if Internet servers that receive a large amount of traffic are known, any number of servers can be used to provide the functionality of the server 605 described herein and any one of a variety of load balancing techniques can be used It balances the traffic load from the client device 603 across these servers.

In an embodiment in which a user supplies a search query by voice (ie, in the form of audio data), many techniques can be used to facilitate automatic voice recognition of audio data to generate search queries. These techniques can be used in both the embodiment in which the automatic voice recognition is executed on the client device and the embodiment in which the automatic voice recognition is executed on a computer outside the client device.

In some embodiments, the language model used by the automatic voice recognizer in performing voice recognition on the audio data of the search query provided to the user can be selected based on the search engine(s) to be queried. For example, as shown in FIG. 7, in some embodiments, the automatic speech recognizer may use multiple different language models to perform recognition on the same audio data to generate multiple different recognition results. In FIG. 7, speech recognition is performed on the audio data 701 using language models 703a, 703b, ..., 703n to generate recognition results 705a, 705b, ..., 705n. Depending on the difference between the content of the audio data 701 and the language model 703, each of the recognition results 705 may be the same, each of the recognition results 705 may be different from each other, or some of the recognition results 705 may be the same and the recognition results 705 may be the same. Some may be different. One or more of the language models 703 can be associated with a specific search engine and the recognition results generated using a language model associated with a specific search engine can be used as the basis for search queries sent to the search engine.

Therefore, for example, in some embodiments, a language model that has been trained based on content indexed by a particular search engine can be used to generate recognition results that are used as the basis for queries sent to the search engine. This makes it more likely to generate a query related to the search engine's search results.

This technique can be especially useful when sending queries to a site-specific search engine that indexes a limited amount of content (which is generally about a specific subject area). For example, an online shoe retailer may have a site-specific search engine on one of its e-commerce websites. Therefore, the content of this website may include shoe-specific terms and phrases, such as shoe brand and model names, descriptions of shoe attributes, and various other shoe-specific information. By training the language model used by a voice recognizer based on the content of this website, the recognizer is more likely to accurately recognize voices that include shoe-specific terms and phrases.

Therefore, for example, if a user-supplied search query in the form of an audio input is received and to be sent to both a site-specific search engine and a general search engine, the audio input can use a first language model (e.g., A language model that has been trained based on content related to content indexed by the site-specific search engine) to generate an identification result that is used as one of the basis for the query to be sent to the site-specific search engine and can also be used A second language model that is different from the first language model (for example, it has been trained based on more general content) is recognized to generate a recognition result that is used as a basis for a query to be sent to a general search engine.

In the above example, the recognition result obtained by performing voice recognition using a language model associated with a specific search engine is used as the basis for the search query sent to the search engine. However, in some embodiments, instead of sending the recognition results obtained using different language models to different search engines, different language models may be used to obtain multiple recognition results, and the recognition result with the highest score or credibility value can be used as the One of the basis for search queries in a search engine. For example, in FIG. 7, each of the recognition results 705 can be generated by performing automatic voice recognition on the audio data 701 using a specific language model. In some embodiments, one of the identification results 705 may be selected as the result used as the basis for generating a search query to be sent to one of the plurality of search engines. This choice can be made in any suitable way. For example, as discussed above, the recognition result with the highest score or credibility value due to voice recognition may be selected or any other appropriate criterion or combination of criteria may be used.

The inventors have recognized that in a scenario where multiple language models are used to perform voice recognition on the same audio data, using each of different language models to perform multiple separate and independent voice recognition procedures can increase the delay in obtaining recognition results and /Or the amount of processing resources used in performing voice recognition can be increased.

Therefore, in some embodiments, instead of performing an entire independent speech recognition process on the audio data for each language model, a general language model can be used to generate a lattice of hypothetical word sequences, and a lattice can be used. One or more selected language models re-score this grid to generate recognition result(s) specific to the selected language model(s). In this way, the time and processing resources consumed in generating the recognition results can be reduced, because the same grid is used to generate each recognition result and therefore only needs to be generated once.

In the example of FIG. 7, the audio data 701 is shown as each directly provided to the language model 703. This description is only provided to facilitate the understanding of how multiple language models can be used, and it should be understood that, in practice, language models may not directly manipulate audio data. For example, a sound model and a universal language model can be used to obtain a grid with multiple hypothetical word sequences from audio data, and a dedicated language model 703 can be used to re-score the universal grid and obtain a specific search engine A sequence of words.

In some embodiments, instead of using multiple different language models in generating search queries to be provided to the search engine, a language model can be selected from a plurality of available language models. For example, in some embodiments, a language model can be recognized as being very suitable for recognizing a specific voice query, and the language model can be used to recognize the query.

Any of a variety of possible criteria or combinations of criteria can be used to select a language model to be used to recognize a particular voice query. For example, in some embodiments, the content of the query can be used as a basis for selecting a language model to be used. For example, a two-stage process can be used, whereby in a first stage, a general language model that does not have a dedicated vocabulary is used to perform automatic voice recognition of a voice query or a part of a voice query. This language model can recognize some words in the query, but it cannot accurately recognize all words. The recognition results obtained from the first stage can be used to classify queries based on topics or categories. Any of a variety of well-known classification techniques can be used.

Based on the classification, a language model with a dedicated vocabulary for one of the recognized topics or categories can be selected. After the language model is selected, the selected language model can be used to perform a second stage of automatic speech recognition. For example, if it is determined after the first stage that the voice query contains words such as "direction", "street" or "road", it can be determined that the voice query is a map-related query, and a map-related language model (for example, including the United States And/or a language model of city names, street names, and place names of other countries) can be selected and used in the second stage.

As discussed in more detail below, the recognition results from the second stage can be used as the basis for one or more search queries to be provided to one or more search engines. In addition, as discussed in more detail below, the search engine or search engines to be queried can be selected based on the language model selected to recognize the query in the second stage or the content of the recognition result obtained from the second stage.

In some embodiments, a language model for identifying a voice search query supplied by a user as an audio data can be updated based on the topic of interest. This can be done in any of a number of possible ways. The inventors have recognized that current events often cause search engines to receive a large number of search queries (including keywords associated with a specific event, including keywords of subjects that have not been used previously and are not frequently searched). For example, if a popular new movie is played in a movie theater, the search engine can receive a large number of search queries from different users that include the name of the movie, the name of the actors in the movie, or other keywords associated with the movie. Similarly, if a major sporting event is coming, the search engine can receive a large number of queries including the names of the teams and/or athletes participating in the event, or an event in popular culture can include people previously unknown to the general public. Put it into the focus of public attention (for example, the person who won the prize, the person accused of a high-profile crime, the person involved in a high-profile scandal).

In some embodiments, the words and/or phrases associated with the topics of the recent user's attention can be identified, and the language model can be updated to improve the accuracy of recognizing these words and/or phrases. These words and/or phrases can be recognized in any of a variety of ways. For example, in some embodiments in which the user-provided query (for example, in audio format, text format, or some other format) is provided from a plurality of client devices to a server or server set, the server or server set Frequently used search terms or phrases can be identified in these search queries and the language model can be updated based on these frequently used search terms. In other embodiments, one or more persons may be manually identified by one or more persons assigned to the task of identifying words and/or phrases associated with the topic of interest.

The language model can be updated in this way with any degree of frequency desired. For example, in some embodiments, the language model may be updated at regular intervals, such as once a day, once every two days, twice a day, once a week, or any of a variety of other possible intervals. In some embodiments, the language model may be updated in response to a user action (such as an instruction from the user to update the language model) or in any other suitable manner.

In some embodiments, the words provided in the user's search query, words obtained from web crawling, and/or words from other sources can be collected and used to update the language model. It is possible to maintain a word count indicating the number of occurrences of the word in the source data, and to add words whose word count exceeds a threshold value to the language model.

In some embodiments, an automatic pronunciation (for example, a phoneme sequence) of each word to be added to the language model can be generated and included in the language model with its corresponding word. In some embodiments, a human reviewer (e.g., a linguist) can review and (if approved) modify the automatically generated pronunciation before it is added to the language model.

In some embodiments, feedback indicating which search results a user chooses can be used to update the language model used in the automatic speech recognition of search queries. This can be done in any of a number of possible ways. For example, in some embodiments, an automatic voice recognizer can generate a recognition result from the audio data of a voice search query provided by a user. This identification result can be used as a basis for one or more search queries provided to one or more search engines, and search results can be returned from the one or more search engines and used in response to the query or queries Show those search results. The user can select one or more of the search results to observe a specific webpage or content piece. In some embodiments, information about which search result(s) the user selects can be used to update the language model used by the automatic speech recognizer. For example, a user may speak the search query "Willie Mays" and the automatic voice recognizer may incorrectly recognize the voice as "Willy Maze". Therefore, a search query for the phrase "Willy Maze" can be sent to one or more search engines. However, one or more of the search engines may return a hyperlink to a web page with biographical information about Willie Mays in its result list. If the user selects this hyperlink to access the webpage, the content of the webpage can be used to update the language model. This can increase the possibility that the phrase "Willie Mays" will be correctly recognized next time a user speaks.

In some embodiments, a previous speech obtained from a specific user can be used to customize the language model for that user into the user's voice. This can be done in any of a variety of ways. For example, in some embodiments, the goal is to find a set of representative latent speakers to "interpret" the voice data received from a specific speaker, which can be expressed as a bag-of-word feature vector . The result is, for example, clustering similar speakers together in a probabilistic manner. These clusters can be used to construct a set of latent speaker language model components. Using these latent components, a specific user-specific adaptation is performed by estimating the speaker-specific linear interpolation weights of the language model components using the speech data received from the specific speaker performing the adaptation.

In some embodiments, the language model for a specific user can be updated based on the user's historical browsing information. This can be done in any of a variety of ways. For example, based on the user's browsing history, frequently visited web pages can be identified. A training corpus that emphasizes content from frequently visited web pages can be used to retrain the language model for recognizing voice queries.

Any one or the owner of the above technologies for selecting a language model to facilitate automatic speech recognition can be used in combination with each other. In addition, it should be understood that there is no need to use these techniques for selecting a language model, and in some embodiments, the same language model can be used to process all speech inputs.

<b>II. Determine which search engines to query</b>

As discussed above, in action 203 in the process 200, in response to receiving a user-specified query, the application determines how many and which search engines to query. This can be done in any of a variety of ways.

For example, in some embodiments, the set of search engines that are queried may be static. For example, the application can be hard-coded or otherwise configured to query a specific set of search engines. In these embodiments, the application queries the same set of search engines in response to each user-provided search query it receives. Therefore, in these embodiments, the action 203 can be regarded as the search engine to be queried by executing the determination of the search engine identified by the static information in the application or the static information that can be accessed by the application. The software code of the application program is executed.

In other embodiments, the determination of which search engines to query may be dynamically made based on one or more of a variety of possible criteria. For example, the search engine to be queried can be determined based on the following: explicit commands from one or more search engines that are instructed by the user; the content of the query; the historical browsing information or access patterns of the user who provides the query; Different from the historical browsing information or access patterns of one or more other users who provided the query; any combination of two or more of the above; and/or any one or more of many other possible criteria By.

In some embodiments, the application running on the client device can use the criteria listed above to determine which search engines to query. In other embodiments, a computer outside the client device running the application can use the criteria listed above to determine which search engines to query, and this computer can provide information identifying the search engine to be queried To the application. In these embodiments, the application determines which search engine to query from the information provided to it by the external computer.

For example, in the graphical explanatory environment in FIG. 6, the server 605 can make a determination as to which search engine(s) to query, and based on this determination, the server 605 can instruct the application to be executed on the client device 603 Which search engine(s) the program queries. In some embodiments, the server 605 can instruct the application program to query which search engine(s) by providing the global resource locator (URL) of the search engine to be queried to the application program, where each search engine to be queried The URL contains a search string appropriately formatted for the search engine. For example, as shown in FIG. 8, the application program executed on the client device 803 can receive a search query from a user 801, and the search query can be in a text format, an audio format, or some other format. The client device 803 can pass user-supplied search queries to the server 805, which can determine which search engines to query; use the appropriate search string to generate the URL of the determined search engine; and return the URLs to The application program on the client device 803. In the context where the user-supplied query is provided as a voice query of audio data, in some embodiments, the server 805 may perform automatic voice recognition on the audio data to generate a recognition result and determine the query based on the recognition result Which search engines. In other embodiments, automatic voice recognition can be performed on the client device 803, and the recognition result can be sent from the client device 803 to the server 805 instead of audio data.

Figure 9 shows an example of the URL of three different search engines that can be generated for the search query "Miles Davis". In an embodiment where the determination of which search engines to query is only made on the client device, the client device can generate URLs for different search engines. In some embodiments where a determination is made on the server as to which search engines to query, the server can instruct the client device which search engines to query and the client device can generate URLs for these search engines.

Some search engines cannot use a simple URL query. For example, for some search engines, first, the client establishes a connection or session with the search engine, and then sends the query to the search engine. If the search engine is identified as a search engine to be queried, a format suitable for the search engine can be used to generate a query based on the content of the query provided by the user. Like the URL, this search engine query can be generated by an application on the client device and/or a computer (for example, the server 605) external to the client device.

As discussed above, in some embodiments, the determination of which search engines to query may be made based at least in part on direct input from a user identifying which search engines he or she desires to query. For example, the user can select which search engine he or she wants to query from a list provided by the application. The user can query (for example, "give me the Wikipedia page for George Washington" or "give me the Yelp review" "for John Doe's Bar and Grill" etc.) or may provide this input in some other way.

In some embodiments, this input can be provided every time the user provides a search query to the application. In other embodiments, the user can provide this input once, and the application can store information identifying the search engine that the user indicates, and each time a user provides a search engine, the application can use the stored information To determine which search engines to query.

In some embodiments, instead of the application program storing this information, the application program can provide this information to an external computer (for example, the server 605 in FIG. 6) and can store the information on the external computer. Therefore, when a user enters a search query into an application, the server 605 can use this information to determine which search engines to query and can instruct the application to query these search engines (for example, by providing a URL or in some other way ).

As discussed above, one or more search engines to be queried can be determined based on the content of the search query provided by the user. In some embodiments, if a user-supplied search query includes the name of a search engine (the user-supplied search query requests content from the search engine), the search engine can be selected as a search engine to be queried. For example, a user can provide the search query "get me the wikipedia page on Miles Davis". Therefore, the search engine of wikipedia.com can be selected as a search engine to be queried.

In some embodiments, one or more search engines to be queried may be determined based on the type of information requested by the user. For example, if the user provides a search query that requests information about shoes, a search engine on the website of an online shoe retailer can be selected as a search engine to be queried. As another example, if a user provides a search query such as "what is the current weather in New York City?", the search engine of a weather website can be selected as a search engine to be queried.

As also discussed above, in some embodiments, the query may be determined based on the historical browsing information or access patterns of the user who sent the query and/or the historical browsing information or access patterns of one or more other users. One or more search engines to query. For example, if the historical browsing information indicates that a user frequently chooses the link returned in the search results from search engine A but rarely chooses the link returned in the search results from search engine B, the search can be Engine A is selected as one of the search engines to be queried, and search engine B cannot be selected.

Similarly, in some embodiments, if other users historical access patterns instruct other users to frequently select links returned in search results from a particular search engine, the search engine can be selected as A search engine to be queried. For example, if the historical access pattern of other users instructs the user to frequently select the weather link from search engine A and less select the weather link from search engine B, then search engine A can be selected as a pending query The search engine.

<b>III. Send the query to the selected search engine</b>

As discussed above, in action 205 of process 200, the application program running on the client device can send a search query to the search engine identified in action 203. As explained above, a computer (for example, a server) external to the application or client device can generate URLs or other types of search queries corresponding to each of the search engines, where each URL or query contains a search word The search string is based on the search query received from the user (for example, the search query received in action 201 of procedure 200) and is appropriately formatted for its corresponding search engine.

In action 205, the application may send a search query (for example, by accessing the URL) to cause each of the selected search engines to perform a search for the search string contained in its corresponding URL.

In the process 200 of FIG. 2, the search query is sent by the application program running on the client device, so that the search result is returned from the search engine to the application program on the client device. This provides the advantage that the search engine being queried does not receive a large number of queries from the same server or IP address. As explained above, if the server sends queries to the search engine on behalf of a larger number of clients, the search engine being queried can receive a large number of queries from a single source and this behavior can be regarded as a single user A huge number of search queries are sent to search engines. Many search engine operators will regard this behavior as an abuse of search engine services and/or potential denial of service attacks and will take actions to prevent search queries from reaching their search engines from one source that sends such a large number of queries. By sending a search query from a client device, the search engine can treat the query as being sent by the client device that caused it to send, and the sending of these queries can be regarded as normal user behavior.

However, in an embodiment where the URL or query is generated by a computer external to the client device (for example, the server 605 in FIG. 6), the computer can send the query; receive the search results from the corresponding search engine; and The search results are returned to the application on the client device.

In some embodiments, in response to receiving a search query from the user in act 201, the query of each of the search engines selected in act 203 may be automatically sent. In other embodiments, until the user provides an input requesting to observe the search results from the owner or some of the search engines selected in action 203, the query of the search engines can be sent to the search engines. For example, search results from some search engines can contain a lot of content. As an example, a search result from a search engine on a website that has video clips may include a thumbnail image of each video included in the result list. Therefore, it can take a relatively long time to transmit the search results from the search engine to the client device or external computer. If the user has never observed the search results from the search engine, the time it takes to send the search results may be unnecessary time. Therefore, in some embodiments, search queries to search engines that return results that include bandwidth-intensive content (such as large amounts of images, video data, audio data, and/or other bandwidth-intensive content) cannot respond to receiving-use The search query can be automatically sent by the provider, and the search query to the search engine that returns results that do not contain bandwidth-intensive content can be automatically sent.

It should be understood that the size of the content of a search result set (or the expected size of the content) can be used to determine whether to automatically send a search query to a specific search engine or whether to wait for a user's instruction before sending the query to the search engine An example of one of the criteria. Any one of a variety of other criteria or a combination of criteria can be used.

<b>IV. Receive search results from search engines</b>

As discussed above, in action 207 of process 200, the application program running on the client device can receive search engine results generated by the search engine queried in action 205. The search engine results can be received in any of a variety of possible ways. In embodiments where the client device sends search queries to these search engines, each search engine can provide its results directly to the client device. In an embodiment in which an external computer (for example, the server 605 in FIG. 6) sends a search query to a search engine, each search engine can provide its results to the external computer, and the external computer can do so The result is provided to the client device.

The results can be received in any of a variety of formats. Some search engines provide results as a hypertext markup language (HTML) webpage that contains a list of hyperlinks that are identified as relevant to the search query and/or webpages. Other search engines can provide results in different formats. It should be understood that the application can receive results in any of a variety of formats and all search results received need not be in the same format.

<b>V. Display search results</b>

In act 209 of the process 200, the application program may cause the search result received in act 207 to be displayed on a display device integrated with or coupled to the client device. The results can be displayed in any of a variety of formats and can be presented in any of a variety of possible ways.

For example, in some embodiments, the search results from the search engine that provides the search results as an HTML web page can be displayed by presenting the HTML code in the web page. This is different from meta search engines (such as Dogpile discussed above)^TM), the display result does not display the HTML code received from a search engine, but retrieves the link from the HTML code and generates a new HTML code to display the retrieved link.

For search results that are not received in HTML format, HTML code can be automatically generated to display the received search results, and the search results can be displayed by presenting the automatically generated HTML code.

In some embodiments, the search results from each search engine can be displayed to be visually separated from the search results from other search engines, so that the user clearly knows which search engine provides each search result when viewing the search results gather. In addition, in some of these embodiments, the order of the search result list returned from each search engine can be stored in the displayed search results of each search engine, so that the user can observe the determination of each search engine How relevant is each search result in its list.

The search results from each search engine can be visually separated from each other in any of a number of possible ways. For example, in some embodiments, search results from multiple search engines can be displayed on the display at the same time, and separated by rows, columns, or in some other way. In some embodiments, at any given time, only one of the search results of the search engine can be displayed on the display, and the user can control the display of the search engine by selecting an icon, button or other user interface element Which one of the search results.

In some embodiments, a rotating disk with a plurality of slots may be displayed on the display device, where each slot corresponds to one of the search engines from which the search results are received. The user's selection of one of the slots of the rotating disk causes the search results from the search engine corresponding to the selected one of the slots to be displayed on the display. FIG. 10 shows an example of a display 1000 using this rotating disk. Figure 10 is from an iPhone available from Apple Computer, Inc. in Cupertino, California^TMAvailable from Dragon Search of Nuance Communications, Inc., Burlington, Massachusetts^TMA screen shot of one of the search results of an application. The screen of FIG. 10 is an example of multiple possibilities that can be used to display search results from multiple search engines.

In FIG. 10, the display 1000 includes a first area in which a rotating disk 1001 is displayed. The rotating disk 1001 includes a number of slots 1003a, 1003b, 1003c, 1003d, and 1003e, each of which corresponds to a search engine (in act 207, the search result is received from the search engine). As discussed above, the search engine corresponding to the slot 1003 can include a general search engine and/or a site-specific search engine, so that the search results provided in response to a user's query can come from multiple different search engines (their Some may be general search engines and some of them may be site-specific search engines).

In some embodiments, the search engine corresponding to each slot can be identified by displaying a symbol, icon, text, image, or other information indicating that the search engine corresponds to the slot in each slot. Because, in some embodiments, the search engine that is queried in response to receiving a user-supplied query at the client device can be dynamically determined, and the search engine assigned to the slot in the rotating disk can also be dynamically determined. That is, for example, for each search engine selected as a search engine to be queried in the action 203 of the procedure 200, a slot is allocated in the carousel. Therefore, it should be understood that in an embodiment where a user (directly or indirectly) specifies which search engines to query, a user can configure which search engines are to be allocated to the slots in the carousel. In addition, in some embodiments, the allocation of the spinning disk slots to the search engines can be made based on the results returned by the specific search engines. For example, if one of these search engines is selected as one of the search engines to be queried at action 203 of the procedure 200, no search result will be returned or an indication that no search result has been found will be returned as a result of the query, Then, one of the slots in the rotating disk may not be allocated to the search engine, and the page returned from the search engine may not be displayed to the user.

The display 1000 may also include an area 1005 in which search results are displayed. The user's selection of one of the slots 1003 in the rotating disk 1001 causes the results received from the search engine corresponding to the selected slot to be displayed in the area 1005. Because the result displayed in the area 1005 when a slot 1003 is selected depends on the query received from the user, the content displayed in the area 1005 may be different for each query sent by different users. That is, for example, when a user selects the slot 1003b in the carousel 1001, the content displayed in the area 1005 for the query "George Washington" is compared with the content displayed in the area 1005 for the query "Miles Davis" The content can be different. Therefore, different content can be displayed for different search queries sent by the user, instead of displaying the same content every time a slot is selected. In this sense, the content displayed in response to a user's selection of a spinning disk slot can be regarded as "dynamic."

The display 1000 may also include an indicator (for example, an arrow or some other type of indicator) 1007 indicating which slot 1003 in the carousel 1001 is currently selected. This indicator identifies which search engine the search result is displayed in the area 1005 for the user.

The display 1000 may also include a query field 1009. The search query displayed in the field 1009 indicates to the user the basis of the search query sent to each of the search engines from which the search results are received. It should be understood from the above discussion that, in some embodiments, different search queries based on the search queries provided by the user can be sent to different search engines. In these embodiments, the query field 1009 may display a user-supplied query that is used as a basis for each of the different search queries sent in response to the query.

A user can enter a search query into the field 1009 in any of a variety of ways. In some embodiments, a user can provide text input to the field 1009 (for example, via a physical or touch screen keyboard). In some embodiments, a user can provide voice input to the field 1009. In the example of FIG. 10, when a user wishes to supply voice input to the field 1009, the user can select the button 1011 and speak his or her search query into a microphone on the client device.

In some embodiments, when the search result is received from the search engine queried in the action 205 of the procedure 200, a rotating disk slot can be selected by default so that the result of a search engine is automatically displayed in the area 1005. The preset spinner slot can be selected in any of a variety of ways. For example, the default spinner slot can be selected based on: which search engine the user observes most frequently; the content of the user's query; which search engine provides the most general results; and/or any other appropriate Guidelines.

In addition, it is not possible to load all the results returned from a specific search engine in response to a query in the area 1005 at one time. Therefore, in some embodiments, a user may have the ability to scroll up and down the search result list and/or adjust the zoom level of the displayed result in order to adjust which part of the search result is displayed in the area 1005.

In addition, the order of the search engine in the rotating disk slot can be selected in any of a variety of ways. In some embodiments, the search engines can be sorted in the carousel based on the following: the frequency of the user's use; the content of the user's query; and/or any other appropriate criteria.

As shown in FIG. 10, due to the size of the display, all slots of the rotating disk 1001 cannot be displayed on the display at one time. For example, in FIG. 10, the slots 1003a and 1003e are only partially shown. In addition, the rotating disk 1001 may include additional slots that are not shown in FIG. 10 at all. In some embodiments, the user can adjust which slots of the rotating disk are displayed by shifting the rotating disk to the right and/or left side of the display. This can be done in any of a variety of ways. For example, the user can use a pointing device (such as a mouse) to drag the rotating disk in one direction. In an implementation in which the display 1000 is displayed on a touch screen, a user can use his or her finger to drag the carousel in one direction.

In some embodiments, the rotating disk 1001 can wrap around the display 1000. Therefore, for example, if the user continuously drags the rotating disk 1001 to the left side of the display 1000, the slot of the rotating disk 1001 moved to the left will move away from the left side of the display and will eventually re-enter the display on the right side. In this regard, dragging the rotating disk 1001 in a specific direction can be regarded as causing the rotating disk to rotate in that direction.

<b>VI. Monitor user actions</b>

As discussed above, when displaying search results from one or more search engines to a user, the user can observe some or all of the search result lists, and can access hyperlinks to observe those results lists Web pages or content identified in.

In some embodiments, the application can monitor user actions taken relative to search results and can store information about the monitored user actions. The monitored action can take any suitable form, because the aspect of the present invention regarding monitoring the user's action is not limited in this respect. In some embodiments, the monitored user actions may include, for example, which search engine the user observes and what web pages and/or content the user accesses.

Subsequently, the information describing the monitored users actions can be used to: update the language model used in the automatic speech recognition of voice queries; determine which search engines to query in response to subsequent user queries; determine how to present search engine results ( For example, determining which carousel slot is preset and/or in which order the search engine is positioned in the carousel slot); and/or various other aspects of facilitating search engine query and/or displaying search engine results.

In an embodiment in which a computer external to the client device uses this information to facilitate any of the above discussed aspects of querying a search engine, the client device can send monitored information to the external computer. The external computer can associate this information with the client device, the user of the client device, and/or the network address that sends the information, and can store it for subsequent use.

<b>VII. Additional implementation details</b>

The computing devices discussed above (eg, client devices, servers, external computers, and/or any other computing devices discussed above) can be implemented in any of a variety of ways. Figure 11 is a block diagram illustrating an illustrative computing device 1100 that can be used to implement any of the computing devices discussed above.

The computing device 1100 may include one or more processors 1101 and one or more tangible non-transitory computer-readable storage media (eg, memory 1103). The memory 1103 can store computer instructions for implementing any of the above-mentioned functions in a tangible non-transitory computer-readable storage medium. The processor(s) 1101 can be coupled to the memory 1103 and can execute these computer instructions to cause the functionality to be realized and executed. The computing device 1100 may also include: a network input/output (I/O) interface 1105. The computing device can communicate with other computers (for example, on a network) via the network input/output (I/O) interface 1105. ) Communication; and one or more user I/O interfaces through which the computer can provide output to a user or receive input from the user. The user I/O interfaces may include devices such as: a keyboard, a mouse, a microphone, a display device (for example, a monitor or touch screen), speakers, a camera, and/or various other Type of I/O device.

The above-described embodiments of the invention can be implemented in any of many ways. For example, hardware, software, or a combination thereof can be used to implement the embodiments. When implemented in software, the software code can be executed on any appropriate processor or collection of processors (whether provided in a single computer or distributed among multiple computers). It should be appreciated that any component or collection of components that perform the above-mentioned functions can be generally regarded as controlling one or more controllers of the functions discussed above. One or more controllers can be implemented in many ways, such as using dedicated hardware or using general-purpose hardware (for example, one or more processors) programmed using microcode or software to perform the functions described above.

In this regard, it should be understood that one implementation of various embodiments of the present invention includes at least one tangible non-transitory computer-readable storage medium (e.g., A computer memory, a floppy disk, a small floppy disk and CD-ROM, a magnetic tape, a flash memory, a field programmable gate array or circuit configuration in other semiconductor devices, etc.), the one or more computer programs When executed on one or more computers or other processors, it performs the functions discussed above in the various embodiments of the present invention. The computer-readable storage medium can be portable so that the program(s) stored thereon can be loaded onto any computer resource to implement various aspects of the invention discussed herein. In addition, it should be understood that the reference to a computer program (which, when executed, performs the functions discussed above) is not limited to an application program running on a host computer. In fact, the term "computer program" used in this article refers in a general sense to any type of computer program code (for example, software Or microcode).

The various aspects of the present invention can be used alone, in combination, or in a variety of configurations that are not specifically discussed in the above-described embodiments, and therefore are not limited to the above-described or diagrams in their applications. The details illustrated in the diagram and the configuration of the components. For example, the aspects described in one embodiment can be combined with the aspects described in other embodiments in any manner.

Furthermore, the embodiments of the present invention may be implemented as one or more methods, and an example of the one or more methods has been provided. The actions to be performed as part of the method(s) can be sequenced in any suitable way. Therefore, embodiments can be constructed in which actions are performed in a different order than the one illustrated in the illustration, and such embodiments may include performing some actions at the same time, even if shown as sequential in the illustrated embodiment.

The use of ordinal terms (such as "first", "second", "third", etc.) in the scope of patent application to modify a claimed element does not mean any priority, priority, or priority of one claimed element over another The sequence of claimed elements or the time sequence of actions to perform a method. These terms are only used as labels to distinguish a claimed element with a specific name from another element with the same name (if ordinal terms are not used).

The wording and terminology used in this article are for descriptive purposes and should not be regarded as limiting. The use of "include", "include", "have", "contain", "involved" and their variants intends to include the items listed thereafter and additional items.

Several embodiments of the present invention have been described in detail, and those skilled in the art will easily think of various modifications and improvements. These modifications and improvements are intended to be within the spirit and scope of the present invention. Therefore, the above description is for example only, and is not intended to be limiting. The present invention is only limited as defined by the scope of the following patent applications and their equivalents.

<p>101. . . Search result collection</p><p>103. . . Search result collection</p><p>105. . . Search result collection</p><p>107. . . Search result collection</p><p>300. . . Network computer environment</p><p>301. . . user</p><p>303. . . Client device</p><p>305. . . Internet</p><p>307a-n. . . Search engine</p><p>401. . . Client device</p><p>403. . . application</p><p>405. . . Automatic Voice Recognizer (ASR)</p><p>407. . . User-supplied audio data</p><p>409. . . Identification result</p><p>501. . . Client device</p><p>503. . . application</p><p>505. . . server</p><p>507. . . Automatic voice recognizer</p><p>509. . . Audio data</p><p>511. . . Respond to</p><p>601a-n. . . user</p><p>603a-n. . . Client device</p><p>607. . . Internet</p><p>609a-n. . . Search engine</p><p>701. . . Audio data</p><p>703a-n. . . Language model</p><p>705a-n. . . Identification result</p><p>801. . . user</p><p>803. . . Client device</p><p>805. . . server</p><p>1000. . . monitor</p><p>1001. . . Rotating disc</p><p>1003a-n. . . Slot</p><p>1005. . . area</p><p>1007. . . indicator</p><p>1009. . . Query field</p><p>1011. . . Button</p><p>1100. . . Computing device</p><p>1101. . . processor</p><p>1103. . . Memory</p><p>1105. . . Network input/output (I/O) interface</p><p>1107. . . User input/output (I/O) interface</p><p>Q. . . Search result</p><p>R. . . Search result</p><p>S. . . Search result</p><p>T. . . Search result</p><p>X. . . Search result</p><p>Y. . . Search result</p><p>Z. . . Search result</p>

Figure 1 shows a Venn diagram of a collection of search results from multiple different search engines;

2 is a flowchart of an illustrative process for sending search queries to multiple search engines and receiving search results from the multiple search engines according to some embodiments;

Figure 3 is a block diagram of a computer environment in which some embodiments can be implemented;

FIG. 4 is a block diagram of a client device (which executes an application program for querying multiple search engines and an automatic voice recognizer for voice recognition for voice search queries) according to some embodiments;

FIG. 5 is a client device (which executes an application for querying multiple search engines) and a server (which executes an automatic voice recognizer and provides voice recognition services to the client device according to some embodiments) A block diagram of performing voice recognition on voice search queries);

FIG. 6 is a block diagram of a computing environment in which some embodiments can be implemented;

FIG. 7 is a block diagram of audio data of a voice query in which multiple different language models are used to recognize a voice query according to some embodiments;

FIG. 8 is a block diagram of a server instructing a client device to query which search engines in response to receiving a user-supplied search query according to some embodiments;

FIG. 9 is a diagram showing a global resource locator (URL) (which can be generated to query multiple search engines) according to some embodiments;

Figure 10 is a diagram of a display of a client device which can display search results from multiple search engines; and

Figure 11 is a block diagram of an illustrative computing device on which one of the modes described below can be implemented.