Analyzing natural language expressions in a data visualization user interface
19 claims: 7 independent, 12 dependent
- 1方法であって、ディスプレイと結合されているコンピューティングデバイスであって、1つ以上のプロセッサおよび前記1つ以上のプロセッサによって実行されるように構成された1つ以上のプログラムを格納するメモリを有する、コンピューティングデバイスにおいて、前記ディスプレイにグラフィカルユーザインターフェースを表示することと、第1の用語を含む第1のフレーズに対応する自然言語入力の一部分を識別し、かつ第2のフレーズに対応する前記自然言語入力の第2の部分を識別する、ユーザから受信された前記自然言語入力を分析することと、前記自然言語入力に対応する自然言語表現を表示し、前記自然言語表現に従ってデータ可視化を表示することであって、前記データ可視化は、折れ線グラフ、円グラフ、棒グラフ、ツリーマップ、散布図、またはマップのうちの1つであることと、前記第1のフレーズにおける前記第1の用語を修正する第2の入力を、前記ユーザから、受信することと、前記第2の入力を受信することに応答して、前記修正された第1の用語を有する前記自然言語入力が有効であるかどうか判定するために依存性データベースにおいて検索を実行することによって、前記第2の部分における前記第2のフレーズが前記第1のフレーズの前記第1の用語に機能的に依存していると判定することと、前記第2の部分における前記第2のフレーズが前記第1のフレーズの前記第1の用語に機能的に依存しているという前記判定に応答して、さらなるユーザ入力なしに、前記第2の入力において受信された前記第1の用語への前記修正に基づいて、前記自然言語表現を有効にするように前記自然言語表現における前記第2のフレーズを更新することと、前記第2の入力に基づいて前記第2のフレーズを更新することに応答して、前記グラフィカルユーザインターフェース上に、前記修正された第1のフレーズおよび前記更新された第2のフレーズを含む更新された自然言語表現を表示することと、前記更新された自然言語表現を表す更新されたデータ可視化を表示することと、を含む、方法。
- 2前記自然言語入力が、前記グラフィカルユーザインターフェースにおけるユーザインターフェースコントロールで受信される、請求項1に記載の方法。
- 3前記自然言語入力が、2つ以上の異なるフレーズを含む、請求項1に記載の方法。
- 4前記第1のフレーズにおける前記第1の用語を修正する前記第2の入力が、前記第1のフレーズにおける前記第1の用語を置き換える第2の用語を含む、請求項1に記載の方法。
- 5前記第1のフレーズにおける前記第1の用語を修正する前記第2の入力が、前記第1のフレーズにおける前記第1の用語を削除することを含む、請求項1に記載の方法。
- 6前記第2の入力を受信する前に、前記第1および第2のフレーズに従って、前記更新されたデータ可視化とは異なる初期データ可視化を表示することをさらに含む、請求項1に記載の方法。
- 7前記第2のフレーズが、前記第1のフレーズのサブ部分を含み、前記第2の入力に基づいて前記第2のフレーズを更新することが、前記第1のフレーズの前記サブ部分を更新することを含む、請求項1に記載の方法。
- 8前記第1のフレーズおよび前記第2のフレーズが、異なるフレーズである、請求項1に記載の方法。
- 9前記第2の用語に基づいて前記第2のフレーズを更新することが、第3の用語を置き換えるために、前記第2のフレーズから前記第3の用語を削除することと、前記第2の用語を前記第2のフレーズに追加することと、を含む、請求項 4 に記載の方法。
- 10前記第2の用語に基づいて前記第2のフレーズを更新することが、前記第2のフレーズを削除することを含む、請求項 4 に記載の方法。
- 11ディスプレイと結合されたコンピューティングデバイスであって、1つ以上のプロセッサと、前記1つ以上のプロセッサによって実行されるように構成された1つ以上のプログラムを格納するメモリと、を含み、前記1つ以上のプログラムが、前記ディスプレイにグラフィカルユーザインターフェースを表示することと、第1の用語を含む第1のフレーズに対応する自然言語入力の一部分を識別し、かつ第2のフレーズに対応する前記自然言語入力の第2の部分を識別する、ユーザから受信された前記自然言語入力を分析することと、前記自然言語入力に対応する自然言語表現を表示し、前記自然言語表現に従ってデータ可視化を表示することであって、前記データ可視化は、折れ線グラフ、円グラフ、棒グラフ、ツリーマップ、散布図、またはマップのうちの1つであることと、前記第1のフレーズにおける前記第1の用語を修正する第2の入力を、前記ユーザから、受信することと、前記第2の入力を受信することに応答して、前記修正された第1の用語を有する前記自然言語入力が有効であるかどうか判定するために依存性データベースにおいて検索を実行することによって、前記第2の部分における前記第2のフレーズが前記第1のフレーズの前記第1の用語に機能的に依存していると判定することと、前記第2の部分における前記第2のフレーズが前記第1のフレーズの前記第1の用語に機能的に依存しているという前記判定に応答して、さらなるユーザ入力なしに、前記第2の入力において受信された前記第1の用語への前記修正に基づいて、前記自然言語表現を有効にするように前記自然言語表現における前記第2のフレーズを更新することと、前記第2の入力に基づいて前記第2のフレーズを更新することに応答して、前記グラフィカルユーザインターフェース上に、前記修正された第1のフレーズおよび前記更新された第2のフレーズを含む更新された自然言語表現を表示することと、前記更新された自然言語表現を表す更新されたデータ可視化を表示することと、を行わせる命令を含む、コンピューティングデバイス。
- 12前記自然言語入力が、2つ以上の異なるフレーズを含む、請求項11に記載のコンピューティングデバイス。
- 13前記第1のフレーズにおける前記第1の用語を修正する前記第2の入力が、前記第1のフレーズにおける前記第1の用語を置き換える第2の用語を含む、請求項11に記載のコンピューティングデバイス。
- 14前記第1のフレーズにおける前記第1の用語を修正する前記第2の入力が、前記第1のフレーズにおける前記第1の用語を削除することを含む、請求項11に記載のコンピューティングデバイス。
- 15前記第2の入力を受信する前に、前記第1および第2のフレーズに従って、前記更新されたデータ可視化とは異なる初期データ可視化を表示させるための命令をさらに含む、請求項11に記載のコンピューティングデバイス。
- 16前記第2のフレーズが、前記第1のフレーズのサブ部分を含み、前記第2の入力に基づいて前記第2のフレーズを更新することが、前記第1のフレーズの前記サブ部分を更新することを含む、請求項11に記載のコンピューティングデバイス。
- 17前記第2の用語に基づいて前記第2のフレーズを更新することが、第3の用語を置き換えるために、前記第2のフレーズから前記第3の用語を削除することと、前記第2の用語を前記第2のフレーズに追加することと、を含む、請求項 13 に記載のコンピューティングデバイス。
- 18前記第2の用語に基づいて前記第2のフレーズを更新することが、前記第2のフレーズを削除することを含む、請求項 13 に記載のコンピューティングデバイス。
- 191つ以上のプロセッサによって実行されるように構成された1つ以上のプログラムを格納する非一時的なコンピュータ可読記憶媒体であって、前記1つ以上のプログラムは、ディスプレイにグラフィカルユーザインターフェースを表示することと、第1の用語を含む第1のフレーズに対応する自然言語入力の一部分を識別し、かつ第2のフレーズに対応する前記自然言語入力の第2の部分を識別する、ユーザから受信された前記自然言語入力を分析することと、前記自然言語入力に対応する自然言語表現を表示し、前記自然言語表現に従ってデータ可視化を表示することであって、前記データ可視化は、折れ線グラフ、円グラフ、棒グラフ、ツリーマップ、散布図、またはマップのうちの1つであることと、前記第1のフレーズにおける前記第1の用語を修正する第2の入力を、前記ユーザから、受信することと、前記第2の入力を受信することに応答して、前記修正された第1の用語を有する前記自然言語入力が有効であるかどうか判定するために依存性データベースにおいて検索を実行することによって、前記第2の部分における前記第2のフレーズが前記第1のフレーズの前記第1の用語に機能的に依存していると判定することと、前記第2の部分における前記第2のフレーズが前記第1のフレーズの前記第1の用語に機能的に依存しているという前記判定に応答して、さらなるユーザ入力なしに、前記第2の入力において受信された前記第1の用語への前記修正に基づいて、前記自然言語表現を有効にするように前記自然言語表現における前記第2のフレーズを更新することと、前記第2の入力に基づいて前記第2のフレーズを更新することに応答して、前記グラフィカルユーザインターフェース上に、前記修正された第1のフレーズおよび前記更新された第2のフレーズを含む更新された自然言語表現を表示することと、前記更新された自然言語表現を表す更新されたデータ可視化を表示することと、を行わせるための命令を含む、非一時的なコンピュータ可読記憶媒体。
Independent claims19
88 paragraphs, as filed
TECHNICAL FIELD Implementations of the present disclosure relate generally to data visualization, and more particularly to systems, methods, and user interfaces that enable users to interact with data visualizations using natural language expressions.
Data visualization applications enable users to visually understand data sets containing distributions, trends, outliers, and other factors important to making business decisions. Some datasets are very large or complex and contain many data fields. Various tools can be used to help users generate data visualizations of various data sets, but typically require users to learn complex user interfaces.
The use of natural language expressions to generate data visualizations provides users with greater convenience of access to data visualization features, including updating fields and changing how data is filtered. Natural language interfaces allow users to develop useful data visualizations with little or no training.
Accordingly, the present disclosure provides more efficient methods and interfaces for manipulating and generating graphical views of data using natural language input. Such methods and interfaces reduce a user's cognitive burden and create a more efficient man-machine interface. For battery-powered devices, such methods and interfaces conserve power and extend battery charging intervals. Such methods and interfaces can complement or replace conventional methods for visualizing data. Other implementations and advantages may be apparent to one of ordinary skill in the art in light of the description and drawings herein.
Some implementations provide automatic updating of relevant phrases in natural language expressions used to generate data visualizations. For example, if the user changes one phrase in the natural language expression, another phrase in the natural language expression may also need to be updated to avoid generating errors. In some implementations, updating the phrase in the natural language expression will change the data visualization representing the data identified by the natural language expression.
According to some implementations, the method is performed on a computing device coupled with a display. For example, a computing device can be a smart phone, tablet, notebook computer, or desktop computer. The method includes displaying a graphical user interface on the display. The method includes analyzing natural language input received from a user to identify a portion of the natural language input corresponding to a first phrase that includes a first term. The method also identifies a second portion corresponding to the second phrase. The method further includes receiving a second input from the user that modifies the first term of the first phrase. In response to receiving the second input, the computing device updates the second phrase based on the second input. In response to updating the second phrase based on the second input, the computing device updates the graphical user interface with the modified first phrase and the updated second phrase. display the updated natural language representation, and display an updated data visualization representing the updated natural language representation.
In some implementations, natural language input is received at a user interface control of a graphical user interface.
In some cases, the natural language input includes two or more different phrases.
Optionally, the second input that modifies the first term in the first phrase includes a second term that replaces the first term in the first phrase.
Optionally, a second input that modifies the first term in the first phrase removes the first term in the first phrase.
Optionally, the method further includes displaying an initial data visualization different from the updated data visualization according to the first and second phrases prior to receiving the second input.
In some implementations, the method further performs a lookup in the database to determine that the second phrase depends on the first term of the first phrase. Updating the second phrase is performed according to a determination that the second phrase depends on the first term of the first phrase.
Optionally, the second phrase is a sub-portion of the first phrase, and updating the second phrase based on the second input updates the sub-portion of the first phrase.
In some cases, the first phrase and the second phrase are different phrases.
Optionally, updating the second phrase based on the second term removes the third term from the second phrase and replaces the second term with the third term to replace the third term. Add to phrase 2.
In some cases, updating the second phrase based on the second term removes the second phrase.
According to some implementations, the method is performed on a computer with a display. For example, the computer can be a smart phone, tablet, notebook computer, or desktop computer. The method includes displaying a graphical user interface on the display. The method includes receiving natural language input from a user specifying filter conditions including a first data field, a relationship, and a comparison value.
The method further includes receiving input to switch from the first data field to the second data field. The method includes, in response to user input, automatically selecting a second comparison value according to the data type of the second data field and updating the data visualization corresponding to the updated filter in a graphical user interface. and displaying on.
Optionally, the domain of the first data field includes the first comparison value.
In some cases, the data type of the first data field is different than the data type of the second data field.
In some implementations, the method further includes displaying on the graphical user interface an initial data visualization that differs from the updated data visualization according to the filter criteria prior to receiving the user update.
In some implementations, the method further includes identifying a default value for the second comparison value.
In some implementations, a computing device includes one or more processors, memory, a display, and one or more programs stored in the memory. Programs are configured to be executed by one or more processors. One or more programs contain instructions for performing any of the methods described herein.
In some implementations, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by a computing device having one or more processors, memory, and displays. One or more programs contain instructions for performing any of the methods described herein.
Thus, methods, systems, and graphical user interfaces are disclosed that allow users to easily build and update data visualizations using natural language commands.
For a better understanding of the disclosed systems, methods, and graphical user interfaces, as well as additional systems, methods, and graphical user interfaces that provide natural language interfaces, like reference numerals refer to corresponding parts throughout the figures. Please refer to the following detailed description in conjunction with the following drawings.
<figref num="1">4 is a graphical user interface according to some implementations;</figref><figref num="2">1 is a block diagram of a computing device according to some implementations; FIG.</figref><figref num="3A">4 provides a series of screenshots of a graphical user interface for updating natural language input according to some implementations;</figref><figref num="3B">4 provides a series of screenshots of a graphical user interface for updating natural language input according to some implementations;</figref><figref num="3C">4 provides a series of screenshots of a graphical user interface for updating natural language input according to some implementations;</figref><figref num="3D">4 provides a series of screenshots of a graphical user interface for updating natural language input according to some implementations;</figref><figref num="3E">4 provides a series of screenshots of a graphical user interface for updating natural language input according to some implementations;</figref><figref num="3F">4 provides a series of screenshots of a graphical user interface for updating natural language input according to some implementations;</figref><figref num="4A">6 provides a series of screenshots of a graphical user interface for updating data visualizations based on changes in natural language input according to some implementations;</figref><figref num="4B">6 provides a series of screenshots of a graphical user interface for updating data visualizations based on changes in natural language input according to some implementations;</figref><figref num="4C">6 provides a series of screenshots of a graphical user interface for updating data visualizations based on changes in natural language input according to some implementations;</figref><figref num="4D">6 provides a series of screenshots of a graphical user interface for updating data visualizations based on changes in natural language input according to some implementations;</figref><figref num="4E">6 provides a series of screenshots of a graphical user interface for updating data visualizations based on changes in natural language input according to some implementations;</figref><figref num="5A">4 illustrates updating filters specified by natural language input according to some implementations.</figref><figref num="5B">4 illustrates updating filters specified by natural language input according to some implementations.</figref><figref num="5C">4 illustrates updating filters specified by natural language input according to some implementations.</figref><figref num="5D">4 illustrates updating filters specified by natural language input according to some implementations.</figref><figref num="6A">4 is a flowchart of a process provided for displaying updated data visualizations, according to some implementations;</figref><figref num="6B">4 is a flowchart of a process provided for displaying updated data visualizations, according to some implementations;</figref><figref num="7">6 provides a flowchart of a process for updating data filters according to some implementations.</figref><figref num="8A">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8B">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8C">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8D">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8E">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8F">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8G">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8H">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8I">4 shows widgets used in natural language interfaces according to some implementations.</figref><figref num="8J">4 shows widgets used in natural language interfaces according to some implementations.</figref>
Reference will now be made to implementations, examples of which are illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without these specific details.
Some methods and devices described herein improve data visualization methods by automatically updating the natural language input used to generate the data visualization. Such methods and devices eliminate manual updating of all relevant phrases in natural language input and provide quick and easy access to data visualizations to reduce user burden. If the user modifies part of the natural language input without updating the relevant part of the input, an error condition may be triggered instead of the updated data visualization. In some implementations, if the user modifies one part of the natural language input, another part of the natural language input, such as filters, needs to be updated as well. This requires the user to understand the dependencies of various parts of the natural language input. The methods and devices described herein automatically update the natural language representation so that relevant portions of the input are automatically detected and updated when the user changes portions of the input.
FIG. 1 shows a graphical user interface 100 for interactive data analysis. User interface 100 displays a schema information area 110, also called a data pane. Schema information area 110 provides data fields that can be selected and used to build data visualizations. In some implementations, the data fields of the schema are divided into range groups (eg, categorical data) and metric groups (eg, quantity) in the schema information area 110 . User interface 100 displays user interface controls 120 . In some implementations, user interface controls 120 receive and/or display natural language input 128 (eg, expressions) from a user. In some implementations, graphical user interface 100 includes data visualization area 112 for displaying data visualizations generated based on inputs 128 in user interface controls 120 .
In some implementations, the type of data visualization can be changed using the view type selector 122 . For example, the data visualization view type selected in Figure 1 is "Bar Chart". "map", "line chart", "pie chart", "scatter plot", "text Additional view types for data visualization are available, such as table (text table), and treemap. In some implementations, data visualizations are generated according to a default view type based on the input. In some implementations, the default view type is the data field selected by the user, as described in US Pat. No. 8,099,674 and US Pat. No. 9,424,318, each of which is incorporated by reference in its entirety. The selection is based on the highest visualization type determined according to the type and/or data values of the data fields selected by the user. For example, if the input is modified, a different type of data visualization is displayed (e.g., as described with reference to Figures 3E and 3F where the data visualization type changes from "bar chart" to "text table"). to). In some implementations, the user specifies the data visualization type as part of the natural language input in user interface controls 120 . For example, a user may say "in a Bar Additional phrases may be entered (eg, typed into control 120) specifying "Chart (in bar graph)". For example, the computing device may parse the user input of "in a Bar Chart" and update the view type selector to the "bar chart" option.
In some implementations, in response to the type of data visualization selected from view type selector 122, the computing device displays a phrase in natural language control 120 that includes the data visualization type. For example, the computing device adds in a bar chart to the natural language representation in response to user selection of bar chart in view type selector 122 .
In some implementations, only view types that make sense for the current representation are offered to the user as options. For example, if a user enters (e.g., types into natural language control 120) "in a map," natural language expression 128 does not contain phrases that conflict with the map data visualization, but the computing device set the view type selector 122 to the default data visualization type, and do not include the "map" view type option in the view type selector 122 dropdown. For example, the view type selector 122 dropdown includes only visualization types that make sense based on the natural language input 128 .
In some implementations, a data field may be specified as a range or metric in the database itself (eg, if the data source is a cube data source). In other implementations, the data visualization application 222 automatically assigns a default role to each data field, either metric or range based on the data type of the data field. For example, numeric fields are used as metrics by default, whereas non-numeric fields (eg, text and date fields) are used as ranges by default. Users can override the assigned default role if desired. For example, a numeric "ID" field may initially be classified as a metric, but the user can reclassify the "ID" field as a range.
Ranges are data fields that organize data into categories (also called "buckets"). For example, if the data source contains data associated with "United States" and the data source contains a data field corresponding to "State", then "State" will be used as the range. . Each range creates a different division within the data visualization, such as a separate bar in a bar chart (eg separate bars for each state). These segments are typically labeled with range headers, with one header for each corresponding range value (eg, each bar may be labeled with the corresponding state name).
A metric is a data field used to measure things like sales, profit, order quantity, and is usually continuous. For example, the range "State" has a fixed set of discrete possible values, but the "Sales Amount" data field can have any value within a wide range. A significant number of records may contain various low sales figures associated with low priced items, and many other records may contain higher sales figures of high priced items. Each metric is typically aggregated into a single value at a level of detail (grouping) according to a selected range (e.g. sales may be aggregated by state) (e.g. by default the metric is aggregated is done).
As shown in FIG. 1, natural language input control 120 is used to enter and display natural language expressions 128 . Natural language processor 228 parsed expression 128 into three different phrases 130-1, 130-2, and 130-3. In some cases, one or more phrases are composed of subphrases.
FIG. 2 is a block diagram illustrating a computing device 200 capable of displaying graphical user interface 100 according to some implementations. Various examples of computing device 200 include desktop computers, laptop computers, tablet computers, and other computing devices with displays and processors capable of executing data visualization application 222 . Computing device 200 typically includes one or more processing units/cores (CPUs) 202 for executing modules, programs, and/or instructions stored in memory 214, thereby performing processing operations. , one or more network or other communication interfaces 204, memory 214, and one or more communication buses 212 for interconnecting these components. Communication bus 212 may include circuitry that interconnects and controls communications between system components.
Computing device 200 includes a user interface 206 with a display device 208 and one or more input devices or mechanisms 210 . In some implementations, the input device/mechanism includes a keyboard. In some implementations, the input device/mechanism includes a soft keyboard optionally displayed on display device 208 to allow the user to press keys displayed on display 208 . In some implementations, display 208 and input device/mechanism 210 comprise touch screen displays (also called touch-sensitive displays).
In some implementations, memory 214 includes high speed random access memory such as DRAM, SRAM, DDRRAM, or other random access solid state memory devices. In some implementations, memory 214 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some implementations, memory 214 includes one or more storage devices remotely located from CPU(s) 202 . Memory 214, or alternatively, non-volatile memory device(s) within memory 214, includes non-transitory computer-readable storage media. In some implementations, memory 214, or computer-readable storage media in memory 214, stores the following programs, modules, and data structures, or a subset thereof: Handles various basic system services; an operating system 216 containing procedures for performing hardware dependent tasks; one or more communication network interfaces 204 (wired or wireless) and such as the Internet, other wide area networks, local area networks, metropolitan area networks, etc. a communications module 218 used to connect the computing device 200 to other computers and devices over one or more communications networks; enabling users to communicate with remote computers or devices over the network; browser 220 (or other application capable of displaying web pages); data visualization application 222; In some implementations, data visualization application 222 also includes: o A graphical user interface 100 for the user to construct visual graphics. In some implementations, the graphical user interface includes a user input module 224 for receiving user input via natural language control 120 . For example, a user may use natural language expressions that identify data fields from one or more data sources 240 (which may be stored on computing device 200 or stored remotely) and/or data source(s). Enter 128 (eg, via control 120). Selected fields are used to define visual graphics. Data visualization application 222 then displays the generated visual graphics on user interface 100 . In some implementations, the data visualization application 222 runs as a standalone application (eg, desktop application). In some implementations, the data visualization application 222 performs the following within a web browser 220 or another application that uses web pages served by a web server: o User input (e.g., natural language input); a data visualization generation module 226 that receives and generates a corresponding visual graphic (also called "data visualization" or "datavis"); o A natural language processor 228 that receives and parses natural language input provided by a user. The natural language processor 228 may also include a dependency determination module 230, which searches for dependencies in the database 240 to determine how certain terms and/or phrases are related (eg, dependent). judge. In some implementations, the natural language processor 228 includes a filter generation module 232 that determines whether one or more filters are associated with the user-modified field. Filter generation module 232 generates one or more filters based on changes to fields; o Widget generation module 234 that generates widgets containing user-selectable options. For example, a "sort" widget is generated in response to a user selecting (eg, hovering) over a sorting field (eg, a natural language term identified as a sorting field). The Sort Widget contains user-selectable options such as "ascending", "descending", and/or "alphabetical" that allow the user to sort the selected fields from the widget in any order. You can easily choose how to sort them. o A widget generation module 234 that generates widgets containing user-selectable options. For example, a "sort" widget is generated in response to a user selecting (eg, hovering) over a sorting field (eg, a natural language term identified as a sorting field). The Sort Widget contains user-selectable options such as "ascending", "descending", and/or "alphabetical" that allow the user to sort the selected fields from the widget in any order. You can easily choose how to sort them. o A widget generation module 234 that generates widgets containing user-selectable options. For example, a "sort" widget is generated in response to a user selecting (eg, hovering) over a sorting field (eg, a natural language term identified as a sorting field). The Sort Widget contains user-selectable options such as "ascending", "descending", and/or "alphabetical" that allow the user to sort the selected fields from the widget in any order. You can easily choose how to sort them.
- Zero or more databases or data sources 240 (eg, a first data source 240-1 and a second data source 240-2) used by the data visualization application 222; In some implementations, data sources are stored as spreadsheet files, CSV files, XML files, flat files, JSON files, or stored in relational databases.
Each of the above-identified sets of executable modules, applications, or procedures may be stored in one or more memory devices and correspond to sets of instructions for performing the functions described above. The modules or programs (i.e., sets of instructions) identified above need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules may be used in various implementations. may be combined or otherwise rearranged. In some implementations, memory 214 stores a subset of the modules and data structures identified above. Additionally, memory 214 may store additional modules or data structures not described above.
Although FIG. 2 shows computing device 200, FIG. 2 is intended more as a functional illustration of the various features that may be present, rather than as a structural overview of the implementations described herein. ing. In practice, items shown separately could be combined and some items could be separated, as recognized by those skilled in the art.
3A-3F provide a series of screenshots of the graphical user interface 100. FIG. A user can interact with natural language control 120 to update representation 128 . Natural language expression 128 in FIG. 3A includes three different phrases 130-1, 130-2, and 130-3. Each phrase is identified separately (e.g., in a separate text box) and labeled "sum of Number of Records" 130-1, "by Region" 130-2 (e.g., " by represents the group by function), and sort Region in alphabetical order (arrange regions alphabetically) 130-3. Typically, each phrase 130 includes one or more terms that identify data fields from data source 240 . The term can be a range or a metric. A natural language input may contain more than one term. An initial data visualization representing natural language input is displayed in a graphical user interface, as shown in FIG. 3A. For example, FIG. 3A shows a bar chart representing the number of records per region sorted alphabetically. A "Region" column header 140 indicates that each row of the bar graph corresponds to a region, and a sort indicator 142 indicates that the rows are currently sorted alphabetically (third phrase 130-3 ).
In some cases, the user selects the first term in the natural language expression (eg, via mouse click, hover, or other input). For example, Figure 3B shows that the user selects the third phrase "sort Region in alphabetical It shows that it is hovering over the term "Region" 144 in "order (sort regions alphabetically)" 130-3. In some implementations, the terms are visually differentiated within the natural language input according to the user's selection. For example, the selected term "Region" 144 is underlined in response to the user hovering over it. In some implementations, widgets are generated (eg, using widget generation module 234) in response to user selections, as shown in FIG. 3C. For example, the widget 150 presents the user with user-selectable options (eg, including alternate terms) to replace the selected term 144 . In Figure 3C the user selects the "Country" option 152 from the widget. Depending on the user's selection, the first term "Region" 144 is transformed into the selected term "Region" 144 in the third phrase "sort [term] in alphabetical order" 130-3. Country 146.
In some cases, a second phrase (or a term within a second phrase) depends on an edited phrase (or an edited term within an edited phrase). The sort field must be compatible with the group by field, so for example the second phrase "by Region" 130-2 becomes the third phrase "sort Region in alphabetical order" 130-3 depends on In some implementations, term and/or phrase dependencies are determined by performing searches in databases that store information about data and how the data are related. In some cases, phrases containing overlapping terms are dependent phrases. For example, if both phrases use the term "Region," the computing device may determine that the phrase is the dependent phrase. Now, when the user replaces the term "Region" with "Country" in the third phrase 130-3, the second phrase "by Region" 130-2 is not updated, the computing device raises an error. If the requested data is not grouped by country, the system cannot sort by country. Instead of raising an error, the computing device automatically determines that "by Region" 130-2 should also be updated to avoid returning an error based on user input. This improves the user experience as the user does not have to manually update the second phrase to prevent errors.
The user's input to replace the term "Region" 144 in the third phrase with the term "Country" 146, following the determination that the second phrase depends on the third phrase, also causes the computing device to display "Region" with "Country" 148 to update the second phrase 130-2. The second phrase is automatically updated by the computing device without user input (e.g., the user does not manually change "by Region" to "by Country" after correcting the first term) . Note that the column headers and sort indicators 154 have not yet been updated in FIG. 3C because the changes have not yet been committed. The resulting updated natural language representation is shown in Figure 3D. Further, FIG. 3D shows an updated data visualization representing the updated natural language representation, where the bar chart bars are now sorted alphabetically by country, as indicated by the updated column headers and sort indicator 156. ing.
Figures 3E and 3F show another example of user input that modifies a term within a phrase, causing the device to update another phrase (eg, dependent on the first phrase). For example, the user input in Figure 3E shows the user hovering over the second phrase "by Country" 130-2. In some implementations, the phrase in the natural language representation can be deleted by user input selecting the 'x' option 158 (e.g., shown in FIG. 3E by the user input prompt hovering over the 'x' option 158). There is (eg, it can be erased). For example, the 'x' option 158 is dynamically generated in response to user input (eg, hovering) over a text box containing a phrase. In this example, the computing device selects the second phrase "by Country" 130-2 and the third phrase "sort Country in alphabetical order (arrange countries in alphabetical order) 130-3 are mutually dependent. Figure 3E shows an initial data visualization of a bar chart representing several records grouped by country and sorted alphabetically.
User input modifies second phrase 130 - 2 by removing the second phrase from natural language representation 128 . In response to deleting the second phrase "by Country" 130-2, the computing device deletes the third phrase "sort Country in alphabetical order" 130-3 by deleting the third phrase. Update. The resulting updated natural language representation is shown in Figure 3F. As shown, the second phrase "by Country" 130-2 is removed and the third phrase "sort Country in alphabetical order" 130-3 is automatically removed by the computing device without user input. be. For example, the user selects only the 'x' option 158 for the second phrase and the computing device determines that the third phrase depends on the second phrase and raises an error. Instead, it automatically deleted the third phrase. Therefore, the user did not have to manually delete the third phrase to correct the phrasing. Figure 3F shows the sum of Also shown is an updated data visualization corresponding to the text table representing the phrase 130-1 "Number of Records". Since there is no data field that specifies grouping, all rows of data are grouped to create a single total.
As shown in the example above, the computing device determines how the first phrase has been modified by the user and updates one or more dependent phrases based on the modifications. In some implementations, the computing device updates the terms of the second phrase based on modifications to the first phrase. In some implementations, the computing device deletes the second phrase based on modifications to the first phrase.
Figures 4A-4E provide a series of screenshots of the graphical user interface 100 that updates the data visualization view type based on changes to natural language input. In some implementations, the computing device automatically generates certain types of data visualizations (eg, using data visualization generation module 226) based on natural language input. In some implementations, modifications to natural language input (eg, received from user input) (eg, via natural language control 120) cause the computing device to determine the type of data visualization presented to the user. to change For example, in FIG. 4A, the natural language input is "average Population" 130-4, "by Country" 130-5, "sort Country in descending order by average Population (sorting countries in descending order of average population) 130-6 contains three different phrases. The view type for data visualization is bar graph. As indicated by column headers 160, the bar chart has a bar for each country and sort indicator 162 indicates that the data is sorted in descending order. FIG. 4B shows user input (eg, hovering) for the third phrase 130-6 "sort Country in descending order by average Population." In response to the user hovering over phrase 130-6, the computing device dynamically generates an 'x' option 164 that the user can select to delete phrase 130-6. In response to the user deleting the third phrase in the natural language input (e.g., by selecting the 'x' option 164), the computing device determines that there are no phrases containing 'sort', so the data visualization default type is map. This works because the natural language input is "average Population" 130-4 and "by Country" 130-5. This is shown in FIG. 4C. In some implementations, the user can change the data visualization from the default type of data visualization. For example, the data visualization default type map is shown in drop-down control 168 . Alternate types of data visualization are presented to the user via drop-down control 168, thereby allowing the user to modify the type of data visualization presented in the graphical user interface.
FIG. 4D shows another example of updating the data visualization. In FIG. 4D, the user enters (eg, by typing) an additional phrase 130-7 into natural language input (eg, via natural language input control 120). FIG. 4E shows the computing device changing the data visualization from the map view in FIG. 4D to the bar graph view in FIG. 4E in response to receiving user input. The bar graph in FIG. 4E represents the updated natural language representation showing the average population by country sorted alphabetically by country. Column headers and sort indicator 170 indicate that each row of the bar graph corresponds to a country and that the rows are sorted alphabetically (eg, as specified by additional phrase 130-7). .
5A-5D illustrate update filters for natural language representation 500 according to some implementations. FIG. 5A shows a natural language input 500 with three phrases 502-1, 502-2, and 502-3 each corresponding to a data field. A filter phrase (such as third phrase 502-3) compares a data field to a specific value or another data field such as [data field][relationship][comparison value]. The most common relationships are =, , >, , <, and . In some cases, phrases 502 of natural language input 500 follow a natural language template (eg, stored in a database of a computing device). Based on the natural language template and natural language processing, the computing device determines how to update (eg, or automatically complete) the phrase based on user input. For example, if the first term of a phrase is category, the template identifies that category filters are appropriate for updating the phrase. As another example, if the first term of the phrase is a numeric data field, the template identifies that a quantitative filter is appropriate to update (eg complete) the phrase.
In some implementations, user input (eg, hovering) within user interface control 120 selects term Country 514 . In response to the user hovering over the term (eg, data field) "Country" 514, the computing device automatically (eg, without user input) correlates the partial input with the template phrase and Set a default value (eg, "Argentina" 516) in the second template field. Specifically, the computing device determines that the user has selected a range (data field Country 514) that requires categorical values for comparison. The default comparison value is the data value of country data field 514 . Thus, selecting the data field Country 514 causes the computing device to complete the phrase template with Argentina 516 . These actions occurred before the screenshot in Figure 5A. As shown in FIG. 5A, a data visualization of a map is shown in a graphical user interface (the map is partially hidden behind a dropdown widget 518), which maps country 514 to "Argentina" 516. , which reflects filtering to .
In FIG. 5A, the user has taken another action (eg, clicking country term 514) to open data field selection widget 518. In FIG. As shown in Figure 5A, the country field option 520 is currently selected. Based on the country 514 filter, the comparison value for "Argentina" 516 makes sense. However, when the user selects population option 522 in FIG. 5B, comparing population to the string "Argentina" becomes meaningless.
In FIG. 5B, user input (eg, hovering) within widget 518 selects term Population 522 . The computing device determines that the selected term is quantitative. In response to user input, the computing device automatically updates comparison operator 536 (eg, "at least" or "") and comparison value 538 of phrase 502-3. In this example, the computing device completes the phrase with a comparison value of 10,000. For quantitative data fields like Population 534, you also need to select an aggregation type (eg SUM, COUNT, or AVERAGE). In this example, the aggregation type defaults to AVERAGE524, which appears as aggregation type 532 in the third phrase 502-3. In some implementations, the aggregation type defaults to what is already specified in other phrases (e.g., "Average Population). Additionally, an updated data visualization corresponding to the bar graph is displayed in the graphical user interface based on the updated phrase.
FIG. 5C shows natural language input including a third phrase Country contains 'South' 550-3 with a first data field Country 552. FIG. The comparison operator 554 is contains and the comparison value is South 556 . In this case, the third phrase 550-3 specifies a filter that restricts data to data where the country name contains the text string South 556. As shown in the data visualization, the map shows "South Africa" 560. In FIG. 5C, the user has performed an action to activate the filter widget 518 and the Country option 562 has been selected.
FIG. 5D shows user input switching from the Country option 562 to the Continent option 564 . The computing device determines that both "Country" and "Continent" store category data. Accordingly, the computing device maintains the comparison operator 554 and comparison value 556 of the category filter. The category filter remains an appropriate filter based on the new data field "Continent" 572 selected (eg, no error occurs).
Figures 6A and 6B illustrate a method 600 of displaying data visualizations according to natural language expressions. Method 600 is also referred to as a process. In some implementations, the method is executed and performed on a computing device coupled with a display, the computing device configured to be executed by one or more processors and one or more processors. and a memory storing one or more programs (602). The method 600 performed by a computing device is optionally governed by instructions stored on non-transitory computer-readable storage media. Instructions are executed by one or more processors of the computing device. Each of the operations illustrated in FIGS. 6A and 6B may correspond to instructions stored in computer memory or a non-transitory computer-readable storage medium (eg, memory 214 of computing device 200). Computer-readable storage media may include magnetic or optical disk storage devices, solid-state storage devices such as flash memory, or one or more other non-volatile memory devices. Instructions stored on a computer-readable storage medium may comprise one or more of source code, assembly language code, object code, or other instruction formats interpreted by one or more processors. Some acts in method 600 may be combined and/or the order of some acts may be changed.
In some implementations, the computing device displays (603) a graphical user interface on the display. For example, a computing device displays graphical user interface 100 shown in FIG.
A computing device analyzes natural language input received from a user to identify a portion of the natural language input corresponding to a first phrase that includes a first term (604). In some implementations, natural language input is received at user interface controls 120 in graphical user interface 100 (606). In some implementations, at least a portion of the natural language input is typed by the user. In some implementations, at least a portion of the natural language input is selected by the user from multiple options provided by the computing device. In some implementations, only a portion of the natural language input is received from the user, and the natural language input is automatically completed by the computing device (e.g., how the computing device processes the natural language input). predict and/or suggest completion). For example, a user may have "sum of Number of Records", "by Region", and "sort" can be entered (e.g., typed), and the computing device, based on the user input, will generate a natural language verb with a default phrase (e.g., "Region in alphabetical order"). Complete your input. In some implementations, the natural language input includes two or more different phrases (608). For example, the natural language input (e.g., expression) shown in Figure 1 consists of three different phrases: "sum of Number of Records", "by Region", and "sort Region in alphabetical order". . In some implementations, graphical user interface 100 distinguishes between two or more different phrases by displaying each phrase in a separate user interface element (eg, text box).
In some implementations, prior to receiving the second input, the computing device displays an initial data visualization that differs from the updated data visualization (610) according to the natural language input. For example, the data visualization (eg, bar graph) shown in FIG. 3A is displayed in a graphical user interface. Data visualizations are displayed according to natural language input (eg, a bar chart showing the number of records by region sorted alphabetically).
The computing device receives (611) a second input from the user that modifies the first term in the first phrase. In response to receiving the second input, the computing device updates (614) the second phrase (in natural language input) based on the second input. In some implementations, the second phrase is automatically updated without user input. In some implementations, prior to updating the second phrase, the computing device indicates (eg, on a graphical user interface) how the second input will update the second phrase. For example, the computing device deletes the first phrase (e.g., "by Country") so that the computing device also (e.g., automatically ) indicates to the user that it should be deleted. This indication shows the user how different phrases depend on (eg affect) each other.
In some implementations, the second input includes a second term that replaces the first term (612). In some implementations, updating the second phrase based on the second term includes removing the third term from the second phrase and adding the second term to the second phrase. , replaces the third term (622). For example, the computing device updates at least part of the second phrase to match the changes to the first phrase. For example, Figures 3B-3D show the first phrase (e.g., "sort Region in alphabetical shows a series of screenshots of a graphical user interface in which the first term (e.g., "Region") of the order") has been modified by replacing the first term with a second term (e.g., "Country"). ing. For example, the user provides a second input in FIG. 3C (eg, selects "Country" from the set of terms presented in a dropdown menu). The selected second term (eg country) modifies (eg replaces) the first term (eg region) in the natural language input. A second term from a user who selects a second term (eg, "Country") to replace the first term (eg, "Region") in a first phrase (eg, "sort Region in alphabetical order"). In response to input 2, the computing device replaces the third term with the second phrase ("by By removing the third term (e.g. Region) from the second phrase (e.g. "Country") and adding the second term (e.g. "Country") to the second phrase (e.g. "by Region") ). Figure 3D shows the resulting natural language input, where the second phrase "by Region has been updated to by Country.
In some implementations, the second input removes (613) the first term of the first phrase. In some implementations, updating the second phrase based on the second term removes the second phrase (624). For example, Figures 3E and 3F show a series of screenshots of a graphical user interface in which the first term in the first phrase (eg, the term "Country" in the phrase "by Country") has been removed. (eg, as indicated by user input selecting the "x" next to the phrase in the phrase text box to delete the phrase). In some implementations, all of the first phrase (eg, including the first term) is deleted by the second input. In some implementations, in response to deleting the first phrase (eg, "by Country"), the computing device updates the second phrase. For example, the computing device may select the second phrase (e.g., "sort country in alphabetical order).
In some implementations, the computing device performs a search within the database to determine that the second phrase is dependent on the first term of the first phrase (616). Updating the second phrase is performed according to a determination that the second phrase depends on the first term of the first phrase. In some implementations, the second phrase depends on the first term of the first phrase if modifying the first phrase without modifying the second phrase results in an error condition. . For example, the computing device updates the second phrase so that the updated natural language input can generate the data visualization.
In response to updating (626) the second phrase based on the second input, the computing device displays the revised first phrase and the updated second phrase on the graphical user interface. Display 628 the updated natural language expression, and display 630 an updated data visualization representing the updated natural language expression. For example, Figure 3A shows an initial data visualization representing natural language input showing the number of records by region, and Figure 3D shows an updated data visualization representing the updated natural language input showing the number of records by country. show. As another example, Figure 3E shows the initial data visualization as a bar graph before the second input, and Figure 3F shows the updated data after the second phrase has been updated in response to the second input. data visualization (for example, a text table showing "40,660").
In some implementations, the second phrase includes a sub-portion of the first phrase, and updating the second phrase based on the second input updates the sub-portion of the first phrase. including (618). For example, a first phrase includes a first term and further includes a second phrase. Accordingly, in response to the second input, the computing device updates another term within the same phrase (eg, the first phrase).
In some implementations, the first phrase and the second phrase are different phrases (620). For example, the examples described above with reference to FIGS. 3A-3F illustrate cases where the first phrase differs from the second phrase (eg, the second phrase is not a sub-part of the first phrase). ).
FIG. 7 illustrates a method 700 for updating natural language expression filter conditions, according to some implementations. Method 700 is also referred to as a process. In some implementations, the method is performed and performed on a computing device coupled with a display (702). A computing device has one or more processors and memory (702). The memory stores one or more programs configured for execution by one or more processors. The method 700 performed by a computing device is optionally governed by instructions stored on non-transitory computer-readable storage media. Instructions are executed by one or more processors of the computing device. Each of the acts illustrated in FIG. 7 may correspond to instructions stored in computer memory or a non-transitory computer-readable storage medium (eg, memory 214 of computing device 200). Computer-readable storage media may include magnetic or optical disk storage devices, solid-state storage devices such as flash memory, or one or more other non-volatile memory devices. Instructions stored on a computer-readable storage medium may comprise one or more of source code, assembly language code, object code, or other instruction formats interpreted by one or more processors. Some acts in method 700 may be combined and/or the order of some acts may be changed.
In some implementations, the computing device displays a graphical user interface on the display (704). For example, a computing device displays graphical user interface 100 shown in FIG.
A computing device receives natural language input from a user specifying a filter including a first data field, a relationship, and a first comparison value (706). In some implementations, natural language input is received at user interface controls 120 of graphical user interface 100 . In some implementations, at least a portion of the natural language input is typed by the user. In some implementations, at least a portion of the natural language input is selected by the user from multiple options provided by the computing device. In some implementations, only a portion of the natural language input is received from the user, and the natural language input is automatically completed by the computing device (e.g., how the computing device processes the natural language input). predict and/or suggest completion). For example, a user can enter a first data field and the computing device automatically generates (eg, appends) a comparison value based on the first field. For example, the natural language input shown in Figure 5A is "with Contains the third phrase 502-3, "Country in Argentina". The first data field corresponds to "Country" and the comparison value is "Argentina". As described with reference to FIG. 5A, in some implementations the natural language input is matched to a natural language template (eg, to identify the type of filter being used).
In this example, the domain of the first data field contains the comparison value (708). For example, the domain of the first data field "Country" consists of country names containing "Argentina".
In some implementations, prior to receiving an update to the filter specification, the computing device displays an initial data visualization different from the updated data visualization that applies the specified filter (710). For example, Figure 5A shows a map data visualization (eg, partially hidden behind a widget).
A computing device receives user input and replaces a first data field in a filter with a second data field (711). In some cases, the second data field has a different data type than the first data field (712). For example, the user replaces the first data field Country 514 with Population 534 as shown in FIG. 5B. In this example, the first data field "Country" is a categorical data type (eg, range), while the data field "Population" is a quantitative data type (eg, metric). Therefore, switching from the first data field to the second data field requires changing the data type from categorical data to quantitative data. It should be appreciated that user input may also switch from quantitative data to categorical data. In some cases, user input does not change the data type of the data used in the filter. For example, Figures 5C and 5D show the user switching from data field "Country" to data field "Continent", both of which have categorical data. In some cases, the comparison value is not updated (eg, comparison value 556 remains 'South' in FIGS. 5C and 5D) following the determination that the switch does not change the type of data. Optionally, if switching to a data field does not change the type of data, the comparison value is updated to terms contained in the domain of the second data field. For example, if the data field was switched from "Country" to "Continent" and the initial comparison value was "Argentina" (e.g. a value not contained in the domain of "Continent"), the updated comparison value would be Changed to "South America".
In response to receiving the update to the first field, the computing device automatically replaces the first comparison value with the second comparison value (714). For example, in FIGS. 5A and 5B, in response to the user input switching from the first data field "Country" to the second data field "Population," the computing device changes the relationship from "in" to "at least" automatically update to 536 and change "Argentina" to 10,000. Accordingly, the computing device updates the relationship and comparison values to match the second data field. For example, the user does not manually update the second field corresponding to the filter after changing the first field. This provides an intuitive way to generate data visualizations without requiring users to understand the difference between categorical and quantitative filters. Instead, the computing device determines which filters (eg, quantitative or qualitative) to apply based on the user's selection of data fields. In this way, when a user switches a data field to a different data type, the computing device automatically updates the corresponding comparison value to prevent errors from occurring. In some cases, the first data field has a quantitative data type (716) and the second data field has a categorical data type (716). In some cases, the first data field has a categorical data type (718) and the second data field has a quantitative data type (718). For example, FIG. 5B shows the user switching from a first data field having a categorical data type (eg, "Country") to a second data field having a quantitative data type (eg, "Population"). .
In some cases, switching from a first data field to a second data field also requires changing (720) the relationships used by the filter. For example, "contains" is a meaningful relationship in a categorical data field, but not in a quantitative data field.
In some implementations, the computing device identifies a default value for the comparison value. For example, the computing device chooses 10,000 as the default value based on the fact that this number compares to the average population. In some implementations, the computing device selects default values based on information stored in databases and/or data sources (eg, using sampling of data values in data fields). In some implementations, the user updates the default values. In some implementations, after the computing device provides default values, the user manually modifies the values in the natural language expression (eg, by changing filters).
The computing device displays an updated data visualization corresponding to the updated filters (722). For example, FIG. 5B shows a bar graph data visualization (such as partially hidden behind a widget) that filters data according to the data field "Population" and a comparison value of 10,000. As shown in FIGS. 5A and 5B, the initial data visualization of FIG. 5A differs from the updated data visualization of FIG. 5B.
8A-8J illustrate widgets used in natural language interfaces according to some implementations. For example, widget generation module 234 dynamically generates widgets that are displayed to the user in graphical user interface 100 . Widgets are generated based in part on natural language input received from a user. The computing device identifies appropriate widget types based on the selected portion of the natural language input. In this manner, the computing device maps portions of the natural language input to analytical concepts to generate widgets corresponding to the analytical concepts. By dynamically generating widgets based on natural language input, users are given the option of changing to meaningful data visualizations given the input.
For example, FIG. 8A shows receiving natural language input with control 120 listing "Segments in descending order by sales." The computing device analyzes (e.g., parses) the natural language input and identifies that a "descending order" 800 corresponds to an analytical concept (e.g., "Segments") that sorts the data. . Accordingly, the computing device generates a widget that provides the user with multiple sorting options, including a descending option 802 , an ascending option 804 , and an alphabetical option 806 . In this example, descending order option 802 has been selected. Therefore, the segments are sorted in descending order of sales, as indicated by the natural language input.
FIG. 8B shows another example of creating a widget. A natural language phrase "top 20 Segments" is received by a computing device. The computing device determines that 'top 20' 808 corresponds to the analytical concept limit. Accordingly, the computing device generates a widget that allows the user to enter limits and choose between an upper limit 810 and a lower limit 812 . With the upper limit 810 selected, the user can enter how many upper values are to be entered in the quantity text box 813 . Here, the natural language phase specifies the top 20 restrictions. Widgets provide a user-friendly way for users to select and set restrictions and modify natural language phrases.
Figure 8C shows an example of a field list widget. For example, the natural language input only includes the field sales 814 . In some implementations, the field list widget comprises a scrollable widget that lists all relevant content dependent fields in area 816 . The widget includes a search box 818 that allows the user to filter the fields in the list, and a data type dropdown 820 for filtering the displayed fields according to data type. In some implementations, the widget includes an aggregation dropdown 822 that allows the user to select an aggregation type. For example, quantitative aggregation can be sum, average, median, total, total distinct, minimum, maximum, or none (no aggregation). Date aggregation includes year, quarter, quarter name, month, month name, day of month, day of month, week, week number, day of week, hour, hour of day, minute, minute of hour, second, second of minute, exact You can specify the date, etc. For example, Figure 8D shows that the "Order Date" field 824 is "week number (week number)". A data type dropdown 826 limits the fields shown in the area 828 and search bar.
8E-8G are examples of date filter widgets. In some implementations, the widget includes three tabs at the top: Specific Values, Relative Dates, and Absolute Dates. The widgets shown in FIGS. 8E-8G are various interfaces of widgets that are displayed depending on which tab is selected. Each tab contains a dropdown for specifying parameters related to a particular type of date filtering.
FIG. 8H shows an example of a quantitative filter. For example, the field "at least $100" is parsed by the computing device to correspond to a quantitative filter. In some implementations, the widget includes three options: "between" 852, "at least" 854, and "at most" 856. Widgets also include user interface elements such as sliders 860 (eg, indicating minimum and/or maximum values for a field) and editable text entry boxes 858 .
Figures 8I and 8J are examples of non-date categorical filters (eg, based on natural language input 862 corresponding to categorical filters). In some implementations, widgets have one or more tabs at the top (eg, specific values 864 and wildcards 870). This widget allows the user to easily select 'All' 866 or 'None' 868 of the data values displayed in a particular list of values 865 . In some implementations, the widget also includes an option to switch to an absolute date filter or a relative date filter, depending on the determination that the categorical field is date.
The terminology used in the description of the invention herein is for the purpose of describing particular implementations only and is not intended to be limiting of the invention. As used in the description of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. As used herein, the terms "comprising" and/or "comprising" specify the presence of the stated features, steps, acts, elements and/or components, but one or more It will be further understood that it does not exclude the presence or addition of other features, steps, acts, elements, components, and/or groups thereof.
The foregoing description, for purposes of explanation, has been described with reference to specific implementations. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teaching. The implementations were chosen and described in order to best explain the principles of the invention and its practical application, thereby allowing those skilled in the art to make various modifications suitable for the invention and the particular uses contemplated. can take full advantage of the most flexible implementations.
30 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2013109725A | Cites | Japan |
| JP2009086837A | Cites | Japan |
| US20170242913A1 | Cites | United States of America |
15 members in 8 offices
Priority claims5
| Document | Office | Kind | Date |
|---|---|---|---|
| 16134892 | United States of America | – | |
| 16134907 | United States of America | – | |
| 201816134892 | United States of America | A | |
| 201816134907 | United States of America | A | |
| 2019047892 | United States of America | W |
Members15
| Document | Office | Kind | |
|---|---|---|---|
| US2020089700A1 | United States of America | A1 | |
| US2020089760A1 | United States of America | A1 | |
| CA3111511A1 | Canada | A1 | |
| WO2020060720A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US10902045B2 | United States of America | B2 | |
| AU2019344461A1 | Australia | A1 | |
| BR112021005061A2 | Brazil | A2 | |
| US11048871B2 | United States of America | B2 | |
| EP3853714A1 | European Patent Office (EPO) | A1 | |
| CN113383312A | China | A | |
| JP2022500751A | Japan | A | |
| AU2019344461B2 | Australia | B2 | |
| JP7202452B2This record | Japan | B2 | |
| CA3111511C | Canada | C | |
| EP3853714B1 | European Patent Office (EPO) | B1 |
12 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Written notification of registration of transferJAPANESE INTERMEDIATE CODE: R350R350 | R350 | |
| Written request for registration of change of nameJAPANESE INTERMEDIATE CODE: R313533S533 | S533 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 7202452
- Application
- 2021513434
Titles2
- Japanese
- データ可視化ユーザインターフェースにおける自然言語表現の分析
- English
- Analysis of Natural Language Expressions in Data Visualization User Interfaces
Classification
- CPC, 4
- G06F9/451
- G06F16/243
- G06F16/2428
- G06F3/048
- IPC, 2
- G06F16 332
- G06F16 338
