Database query system

Background of the Invention

The invention relates to a database querying tool, and specifically to a database querying tool which will guide a user to interactively create syntactically and semantically correct queries.

End user workstations are being physically connected to central databases at an ever increasing rate. However, to access the information contained in those databases, users must create queries using a standardized query language which in most instances is Structured Query Language (SQL). Most information system organizations consider it unproductive to try and teach their users SQL. As a result there is an increasing interest in tools that create SQL for the user using more intuitive methods of designating the information desired from the database. These tools are generally called SQL Generators.

Most SQL Generators on the market today appear to hide the complexities of SQL from the user. In reality, these tools accomplish this by severely limiting the range of information that can be retrieved. More importantly, these tools make it very easy for users to get incorrect results. These problems arise out of the reality that SQL is very difficult to learn and use. Existing technologies designed to shield users from the complexities of SQL can be grouped into three categories: point-and-shoot menus; natural language systems; and natural language menu systems. Each of these three categories of product / technology have architectural deficiencies that prevent them from truly shielding users from the complexities of SQL.

I. LIMITATIONS OF SQL AS AN END USER QUERY LANGUAGE

SQL is, on the whole, very complex. Some information requirements can be satisfied by very simple SQL statements. For example, to produce from a database a list of customer names and phones for New York customers sorted by zip code, the following SQL statement could be used:<img file="EP0803100B1_D0001.tif" />

In this example, the SELECT command defines which fields to use, the WHERE command defines a condition by which database records are selected, and ORDER BY keywords define how the output should be sorted. The FROM keyword defines in which tables the fields are located. Unfortunately, only a relatively small percentage of information required can be satisfied with such simple SQL

Most information needs, even very simple queries, require complex SQL queries. For example, the SQL statement required to generate a list of orders that have more than two products on backorder, is:<img file="EP0803100B1_D0002.tif" />

This SQL statement contains two SELECT clauses, one nested with the other. For a user to know that this information requirement needs an SQL query involving this type of nesting (known as a correlated subquery) implies some understanding by the user of the relational calculus. However, except for mathematicians and people in the computer field, few users have this skill. The following are some examples of database queries that require more complex SQL constructs: <ul id="ul0001" list-style="none"><li><b>GROUP BY:</b> Approximately 75% of all ad hoc queries require a GROUP BY statement in the SQL. Examples include: <ul id="ul0002" list-style="none" compact="compact"><li><i>Show total sales by division.</i></li><li><i>Show January sales of bedroom sets to Milford Furniture.</i></li></ul></li><li><b>Subqueries:</b> The following are examples of database queries that require subquery constructs which appear as nested WHERE clauses in SQL: <ul id="ul0003" list-style="none" compact="compact"><li><i>Show customers that have children under age 10 and do not have a college fund.</i></li><li><i>Show orders that have more than</i> 2 <i>line items on backorder.</i></li></ul></li><li><b>HAVING:</b> The following are examples of database queries that require the HAVING construct: <ul id="ul0004" list-style="none" compact="compact"><li><i>Show ytd expenses by employee for divisions that have total ytd expenses over $15, 000, 000.</i></li><li><i>Show the name and manager of salesmen that have total outstanding receivable of more than $100,000.</i></li></ul></li><li><b>CREATE VIEW:</b> The following are examples of database queries that require the CREATE VIEW syntax: <ul id="ul0005" list-style="none" compact="compact"><li><i>Show ytd sales by customer with percent of total.</i></li><li><i>What percent of my salesmen have total ytd sales under $25000?</i></li></ul></li><li><b>UNION:</b> The following are examples of database queries that require the UNION construct: <ul id="ul0006" list-style="none" compact="compact"><li><i>Show ytd sales for Connecticut salesmen compared to New York salesmen sorted by product name.</i></li><li><i>Show Q1 sales compared to last year Q1 sales sorted by salesman.</i></li></ul></li></ul>

Thus, common information needs require complex SQL that is likely to be far beyond the understanding of the business people that need this information.

A greater problem than the complexity of SQL is that syntactically correct queries often produce wrong answers. SQL is a context-free language, one that can be fully described by a backus normal form (BNF), or context-free, grammar. However, learning the syntax of the language is not sufficient because many syntactically correct SQL statements produce semantically incorrect answers. This problem is illustrated by some examples using the database that has the tables shown in <b>Figs. 1A-G</b>. If the user queries the database with the following SQL query:<img file="EP0803100B1_D0003.tif" />

The following results are produced: <tables id="tabl0001" num="0001"><table frame="all"><tgroup cols="4" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="39.37mm" /><colspec colnum="2" colname="col2" colwidth="39.37mm" /><colspec colnum="3" colname="col3" colwidth="39.37mm" /><colspec colnum="4" colname="col4" colwidth="39.37mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" /><entry namest="col2" nameend="col2" align="center">NAME</entry><entry namest="col3" nameend="col3" align="center">SUM(ORDER_ DOLLARS)</entry><entry namest="col4" nameend="col4" align="center">SUM(QTY_ ORDERED)</entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="left">American Butcher Block</entry><entry namest="col3" nameend="col3" align="right">119284</entry><entry namest="col4" nameend="col4" align="right">22</entry></row><row><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="left">Barn Door Furniture</entry><entry namest="col3" nameend="col3" align="right">623585</entry><entry namest="col4" nameend="col4" align="right">52</entry></row><row><entry namest="col1" nameend="col1" align="center">3</entry><entry namest="col2" nameend="col2" align="left">Bond Dinettes</entry><entry namest="col3" nameend="col3" align="right">51470</entry><entry namest="col4" nameend="col4" align="right">19</entry></row><row><entry namest="col1" nameend="col1" align="center">4</entry><entry namest="col2" nameend="col2" align="left">Carroll Cut-Rate</entry><entry namest="col3" nameend="col3" align="right">53375</entry><entry namest="col4" nameend="col4" align="right">29</entry></row><row><entry namest="col1" nameend="col1" align="center">5</entry><entry namest="col2" nameend="col2" align="left">Milford Furniture</entry><entry namest="col3" nameend="col3" align="right">756960</entry><entry namest="col4" nameend="col4" align="right">48</entry></row><row><entry namest="col1" nameend="col1" align="center">6</entry><entry namest="col2" nameend="col2" align="left">Porch and Patio</entry><entry namest="col3" nameend="col3" align="right">1113400</entry><entry namest="col4" nameend="col4" align="right">89</entry></row><row><entry namest="col1" nameend="col1" align="center">7</entry><entry namest="col2" nameend="col2" align="left">Railroad Salvage</entry><entry namest="col3" nameend="col3" align="right">85470</entry><entry namest="col4" nameend="col4" align="right">28</entry></row><row><entry namest="col1" nameend="col1" align="center">8</entry><entry namest="col2" nameend="col2" align="left">Sheffield Showrooms</entry><entry namest="col3" nameend="col3" align="right">101245</entry><entry namest="col4" nameend="col4" align="right">26</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">9</entry><entry namest="col2" nameend="col2" align="left">Vista Designs</entry><entry namest="col3" nameend="col3" align="right">61790</entry><entry namest="col4" nameend="col4" align="right">25</entry></row></tbody></tgroup></table></tables>

The second column of this report appears to show the total order amount for each customer. However, the numbers are incorrect. In contrast, the following query<img file="EP0803100B1_D0004.tif" />    produces the correct result: <tables id="tabl0002" num="0002"><table frame="all"><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" /><entry namest="col2" nameend="col2" align="center">NAME</entry><entry namest="col3" nameend="col3" align="center">SUM(ORDER_DOLLARS)</entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="left">American Butcher Block</entry><entry namest="col3" nameend="col3" align="right">83169</entry></row><row><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="left">Barn Door Furniture</entry><entry namest="col3" nameend="col3" align="right">129525</entry></row><row><entry namest="col1" nameend="col1" align="center">3</entry><entry namest="col2" nameend="col2" align="left">Bond Dinettes</entry><entry namest="col3" nameend="col3" align="right">51470</entry></row><row><entry namest="col1" nameend="col1" align="center">4</entry><entry namest="col2" nameend="col2" align="left">Carroll Cut-Rate</entry><entry namest="col3" nameend="col3" align="right">53375</entry></row><row><entry namest="col1" nameend="col1" align="center">5</entry><entry namest="col2" nameend="col2" align="left">Milford Furniture</entry><entry namest="col3" nameend="col3" align="right">111240</entry></row><row><entry namest="col1" nameend="col1" align="center">6</entry><entry namest="col2" nameend="col2" align="left">Porch and Patio</entry><entry namest="col3" nameend="col3" align="right">222680</entry></row><row><entry namest="col1" nameend="col1" align="center">7</entry><entry namest="col2" nameend="col2" align="left">Railroad Salvage</entry><entry namest="col3" nameend="col3" align="right">85470</entry></row><row><entry namest="col1" nameend="col1" align="center">8</entry><entry namest="col2" nameend="col2" align="left">Sheffield Showrooms</entry><entry namest="col3" nameend="col3" align="right">101245</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">9</entry><entry namest="col2" nameend="col2" align="left">Vista Designs</entry><entry namest="col3" nameend="col3" align="right">61790</entry></row></tbody></tgroup></table></tables>

Both SQL queries are syntactically correct, but only the second produces correct numbers for the total order dollars. The problem arises from the fact that before performing the selection and totaling functions, the SQL processor performs a cross-product join on all the tables in the query. In the first query above, three tables are used: Customer (a list of customers with customer data); Order (a list of orders with dollar amounts); and Line_Item (a list of the individual line items on the orders). Since the Order table has the total dollars and there are multiple line items for each order, the joining scheme of the SQL processor creates a separate record containing the total dollars for an order for each instance of a line item. When totaled by Customer, this can produce an incorrect result. When the Line_Item table is not included in the query, the proper result is obtained. Unless the users understand the manner in which the database is designed and the way in which SQL performs its query operations, they cannot be certain that this type of error in the result will or will not occur. Whenever a query may utilize more then two tables, this type of error is possible.

Most information systems users would be reluctant to use a database query tool that could produce two different sets of results for what to them is the same information requirement (i.e. total order dollars for each customer). Virtually every known database query tool suffers from this shortcoming.

A more formal statement of this problem is that the set of acceptable SQL statements for an information system is much smaller than the set of sentences in SQL. This smaller set of sentences is almost certainly not definable as a context-free grammar..

II. POINT-AND-SHOOT QUERY TOOLS

Most SQL generator products are "point-and-shoot" query tools. This class of products eliminates the need for users to enter SQL statements directly by offering users a series of point-and-shoot menu choices. In response to the user choices, point-and-shoot query tools create SQL statements, execute them, and present the results to the user, appearing to hide the complexities of SQL from the user. Examples of this class of product include Microsoft's Access, Gupta's Quest, Borland's Paradox, and Oracle's Data Query.

Although such products shield users from SQL syntax, they either limit users to simple SQL queries or require users to understand the theory behind complex SQL constructs. Moreover, because they target the context-free SQL grammar discussed above, it is easy and common for users to get incorrect answers. A point-and-shoot query tool is illustrated below with several examples showing a generic interface similar to several popular query tools representative of this genre. The screen of <b>Fig. 2A</b> appears after the user has chosen the customer table of <b>Fig. 1A</b> out of a pick list. This screen shows the table chosen and three other boxes, one for each of the SELECT, WHERE, and ORDER BY clauses of the SQL statement. If the user selects either the "Fields" box or the "Sort Order" box, a list of the fields in the customer table appears. The user makes choices to fill in the "Fields" and "Sort Order" boxes. In this example, the user chooses to display the NAME, STATE, and BALANCE fields, and to sort by NAME and STATE. This produces the screen of <b>Fig. 2B.</b>

At any time, the user can choose to view the SQL statement that is being created as shown in <b>Fig. 2C</b>. There is a one-to-one correspondence between user choices and the SQL being generated. To fill in the WHERE clause of the SQL statement being compiled, the user chooses the "Conditions" box and fills in the dialog box of <b>Fig. 2D</b> to enter a condition. This produces the completed query design shown in <b>Fig. 2E.</b> The user then chooses the "OK" button to run the query and see the results shown in <b>Fig. 2F.</b>

For queries that involve only a simple SELECT, WHERE, and ORDERBY statement for a single table, a user can readily create and execute SQL statements without knowing SQL or even viewing the SQL that is created.

Unfortunately, only a small proportion of user queries are this simple. Most database queries involve more complex SQL. To illustrate this point, consider a user who wishes to see the same information as in the above example, but to limit the data retrieved to customers of salespersons with total outstanding balance of all the salesperson's customers greater then $80,000. If the user realizes that this query requires two additional SQL clauses (a GROUP BY clause and a HAVING clause) the query (shown in <b>Fig. 2G</b>) can be readily constructed. However, few users are sufficiently familiar with SQL to do so.

Most point-and-shoot query tools cannot handle other complex SQL constructs such as subqueries, CREATE VIEW and UNION. They offer no way (other than entering SQL statements directly) for the user to create these other constructs. Those products that do offer a way to generate other complex constructs require the user to press a "Subquery" or "UNION" or "CREATE VIEW" button. Of course, only users familiar enough with the relational calculus to know how to break a query up into a subquery or use another complex SQL construct would know enough to press the right buttons.

Additional complexity is introduced when data must be retrieved from more than one table. As shown in <b>Fig. 2H,</b> the user may be required to specify how to join the tables together. The typical user query will involve at least three tables. Problems that can arise in specifying joins include: <ul id="ul0007" list-style="none" compact="compact"><li>the columns used to join tables may not have the same name;</li><li>the appropriate join between two tables may involve multiple columns;</li><li>there may be alternative ways of joining two tables; and</li><li>there may not be a way of directly joining two tables, thereby requiring joins through other tables not otherwise used in the query.</li></ul>

In summary, point-and-shoot query tools shield users from syntactic errors, but still require users to understand SQL theory. The other critical limitation of point-and-shoot menu products is that they target the context-free SQL language discussed above. A user seeking total order dollars could as easily generate incorrect SQL statement (3) as correct SQL statement (4) above. Thus, these products generate syntactically correct SQL, but not necessarily semantically correct SQL. Only a user that understands the relational calculus can be assured of making choices that generate both syntactically correct and semantically correct SQL. However, most information system users do not know relational calculus. Moreover, when queries require joins, there are numerous way of making errors that also produce results that have the correct format, but the wrong answer..

III. NATURAL LANGUAGE QUERY TOOLS

Natural language products use a different approach to shielding users from the complexities of SQL. Natural language products allow a user to enter a request for information in conversational English or some other natural language. The natural language product uses one mechanism to deduce the meaning of the input, a second mechanism to locate database elements that correspond to the meaning of the input, and a third mechanism to generate SQL.

Examples of natural language products include Natural Language from Natural Language Inc. and EasyTalk from Intelligent Business Systems (described in U.S. Patent No. 5,197,005 to Shwartz, et al.).

<b>Fig. 3A</b> shows a sample screen for a natural language query system which shows a user query, the answer, another query requesting the SQL, and the SQL.

The sequence of interaction is: <ul id="ul0008" list-style="none" compact="compact"><li>(1) The user types in a free-form English query ("What were the 5 most common defects last month?").</li><li>(2) The software paraphrases the query so that the user can verify its correctness ("What were the 5 defects that occurred the most in June, 1991?").</li><li>(3) If there are spelling errors or if the user query contains ambiguities, the software interacts with the user to clarify the query (not needed in above example).</li><li>(4) The software displays the results.</li></ul>

The attraction of a natural language query tool is that users can express their requests for information in their own words. However, they suffer from several shortcomings. First, they only answer correctly a fraction of the queries a user enters. In some cases, the paraphrase is sufficient to help the user reformulate the query; however, users can become frustrated seeking a formulation that the system will accept. Second, they are difficult to install, often requiring months of effort per application and often requiring consulting services from the natural language vendor. One of the biggest installation barriers is that a huge number of synonyms and other linguistic constructs must be entered in order to achieve anything close to free-form input.

As a compromise, many natural language vendors recommend that, during installation, specific queries are coded and made available to users via question lists. For example, <b>Fig. 3B</b> shows a simple screen containing a list of predefined queries. Users can choose to run queries directly from the list or make minor modifications to the query before running it. Of course, the more they change a query, the more likely it is that the natural language system will not understand the query.

To illustrate the operation natural language products, the architecture of the natural language system described in Shwartz, et al. is used as an example. The system architecture is shown in <b>Fig. 4.</b> The Meaning Representation is the focus of Shwartz et al. The Meaning Representation of a query is designed to hold the meaning of the user query, independent of the words (and language) used to phrase the query and independent of the structure of the database.

The same Meaning Representation should be produced whether the user says "Show total ytd sales for each customer?", "What were the total sales to each of my client's this year to date?", or "Montrez les vendes....." (French). Moreover, the same Meaning Representation should be produced whether: (1) there is a field that holds ytd sales in a customer table in the database; (2) each individual order must be searched, sorted, and totaled to compute the ytd sales for each customer; or (3) ytd sales by customer is simply not available in the database.

The primary rationale for this architecture is that it provides a many-to-one mapping of alternative user queries onto a single canonical form. Many fewer inference rules are then needed to process the canonical form than would be needed to process user queries at the lexical level. This topic is addressed in more detail in Shwartz, "Applied Natural Language", 1987.

The NLI (Natural Language Interface) is responsible for converting the natural language query into a Meaning Representation. The Query Analyzer itself contains processes for syntactic and semantic analysis of the query, spelling correction, pronominal reference, ellipsis resolution, ambiguity resolution, processing of nominal data, resolution of date and time references, the ability to engage the user in clarification dialogues and numerous other functions. Once an initial Meaning Representation is produced, the Context Expert System analyzes it and fills in pronominal referents, intersentential referents, and resolves other elliptical elements of the query. See S. Shwartz, for a more detailed discussion of this topic.

The Meaning Representation for the query "Show ytd sales dollars sorted by salesrep and customer" would be: <ul id="ul0009" list-style="none" compact="compact"><li>SALES: TIME (YTD), DOLLARS, TOTAL</li><li>SALESMAN: SORT(1)</li><li>CUSTOMER: SORT(2)</li></ul>

Again, this meaning representation is independent of the actual database structure. The Database Expert takes this meaning representation, analyzes the actual database structure, locates the database elements that best match the meaning representation, and creates a Retrieval Specification. For a database that has a table, CUSTOMERS, that contains a column holding the total ytd sales dollars, YTD_SALES$, the Retrieval Specification would be: <ul id="ul0010" list-style="none" compact="compact"><li>CUSTOMER.YTD_SALES$:</li><li>SALESMAN.NAME: SORT(1)</li><li>CUSTOMERS.NAME: SORT(2)</li></ul>

The Retrieval Specification would be different if the YTD_SALES$ column was in a different table or if the figure had to be computed from the detailed order records.

The functions of the NLI (and Context Expert) and DBES are necessary solely because free-form, as opposed to formal, language input is allowed. If a formal, context-free command language was used rather than free-form natural language, none of the above processing would be required. The Retrieval Specification is equivalent to a formal, context-free command language.

The Navigator uses a standard graph theory algorithm to find the minimal spanning set among the tables referred to in the Retrieval Specification. This defines the join path for the tables. The MQL Generator then constructs a query in a DBMS-independent query language called MQL. The SQL Generator module then translates MQL into the DBMS-specific SQL. All of the expertise required to ensure that only syntactically and semantically valid SQL is produced is necessarily part of the MQL Generator module. It is the responsibility of this module to reject any Retrieval Specifications for which the system could not generate syntactically and semantically valid SQL.

IV. NATURAL LANGUAGE MENU SYSTEMS

A Natural Language Menu System is a cross between a point-and-shoot query tool and a natural language query tool. A natural language menu system pairs a menu interface with a particular type of natural language processor. Rather than allowing users to input free-form natural language, a context-free grammar is created that defines a formal query language. Rather than inputting queries through a command interface, however, users generate queries in this formal language one word at a time. The grammar is used to determine all possible first words in the sentence, the user chooses a word from the generated list, and the grammar is then used to generate all possible next words in the sentence. Processing continues until a complete sentence is generated.

A natural language menu system will provide a means of ensuring that the user only generates syntactically valid sentences in the sublanguage. However, it can only guarantee that these sentences will be semantically valid for the class of sublanguages in which all sentences are semantically valid. Another difficulty with this class of tool is that it is computationally inadequate for database query. The computational demands of the necessarily recursive algorithm required to run the grammar are immense. Moreover, if the grammar is sufficient to support subqueries, the grammar would probably have to be a cyclic grammar, adding to the computational burden. Finally, the notion of restricting users to a linear sequence of choices is incompatible with modem graphical user interface conventions. That is, users of this type of interface for database query would object to being forced to start with the first word of a query and continue sequentially until the last word of a query. They need to be able to add words in the middle of a query without having to back up and need to be able to enter clauses in different orders.

EP-A-0 287 310 and US-A-4,688,195 disclose such database query systems according to the natural language concept.

From the article "Proceedings of third international conference on data and knowledge bases; improving useability and responsiveness", Jerusalem, Israel; 28 - 30 June 1988, San Matheo, CA., U.S.A., Morgan Kaufmann, U.S.A., pages 3 - 18, Jakobson G. et al: "CALIDA: a system for integrated retrieval from multiple heterogeneous databases." there is known a database query system according to the preamble of claim 1.

Summary of the Invention

It is an object of the invention to improve the known database query system according to the preamble of claim 1 in such a way, that a multitude of databases can be retrieved without the necessity of having detailed knowledge or experience in databases.

The above object is solved by means of a database query system having the features of claim 1, preferred embodiments are defined in the dependent subclaims.

In particular, the claimed database query system allows a dynamic query, based on the information provided by the database. In contrast to the prior art, where a predetermined intermediate language is used according to the present invention the intermediate database language is built up during the creation of a query, i.e. in dependency of the query and the database.

The drawbacks of the prior art are overcome by the system and method of the present invention, which hides the complexity of SQL from the user without limiting the range of information that can be retrieved. Most importantly, incorrect results are avoided

In accordance with the principles of the invention, a Query Assistant is provided that permits the user to enter only queries that are both syntactically and semantically valid (and that can be processed by the SQL Generator to produce semantically valid SQL). The user is never asked to rephrase a query entered through the Query Assistant. Through the use of dialog boxes, a user enters a query in an intermediate English-like language which is easily understood by the user. A Query Expert system monitors the query as it is being built, and using information about the structure of the database, it prevents the user from building semantically incorrect queries by disallowing choices in the dialog boxes which would create incorrect queries. An SQL Generator is also provided which uses a set of transformations and pattern substitutions to convert the intermediate language into a syntactically and semantically correct SQL query

The intermediate language can represent complex SQL queries while at the same time being easy to understand. The intermediate language is also designed to be easily converted into SQL queries. In addition to the Query Assistant and the SQL Generator an administrative facility is provided which allows an administrator to add a conceptual layer to the underlying database making it easier for the user to query the database. This conceptual layer may contain alternate names for columns and tables, paths specifying standard and complex joins, definitions for virtual tables and columns, and limitations on user access.

Brief Description of the Drawings

<b>Figs. 1A to 1G</b> are tables of a sample database used in the examples in the specification.

<b>Figs. 2A to 2H</b> are typical screen displays for a point-and-shoot query tool.

<b>Fig. 3A</b> is a typical screen display for a natural language query tool.

<b>Fig. 3B</b> is a typical screen display for a natural language database query tool with predefined queries.

<b>Fig. 4</b> is a block diagram of the high level architecture of a natural language query tool.

<b>Fig. 5</b> is a block diagram of the high level architecture of the invention.

<b>Fig. 6</b> is a graphic depiction of the tables in <b>Figs. 1A-1G</b> and their relationships.

<b>Fig. 7</b> is a block diagram of the Query Assistant.

<b>Fig. 8</b> is a flow chart of the flow of control of the Query Assistant User Interface.

<b>Fig. 9</b> is a depiction of the initial screen of the user interface.

<b>Figs. 10A to 10G</b> are depictions of dialog boxes used to interact with the user to build a query using the Query Assistant.

<b>Figs. 11A and 11B</b> are a flow chart depicting the flow of control of the SQL Generator.

Detailed Description

1. OVERVIEW

<b>Fig. 5</b> shows a high level block diagram of an intelligent query system that embodies the principles of the invention. It is composed of two parts, the Query System <b>1</b> and Conceptual Layer <b>2</b>. Conceptual Layer <b>2</b> is composed of information derived from database <b>3</b>, including table and column information, and information entered by an administrator to provide more intuitive access to the user. Query System <b>1</b> uses the information from Conceptual Layer <b>2</b> as well as general knowledge about SQL and database querying to limit the user in building queries to only those queries which will produce semantically correct results.

Query System <b>1</b> is further composed of two main components: Query Assistant <b>10</b> and the SQL Generator <b>20</b>. Users create queries using the menu-based Query Assistant <b>10</b> which generates statements in an intermediate query language that take the form of easy to understand sentences. SQL Generator <b>20</b> transforms the intermediate language into a target language (in the illustrated embodiment, SQL). To fulfill the requirement that a user never be asked to rephrase (or reconstruct) a query, the expertise concerning what is and what is not a valid SQL query is placed in Query Assistant <b>10.</b>

SQL Generator <b>20</b> does not contain this expertise. Although users can pose queries directly to SQL Generator <b>20</b>, there is no assurance that semantically valid SQL will be produced. It is logically possible to put some of this expertise into SQL Generator <b>20</b>. However, to assure users that only valid SQL would be generated would require natural language capabilities not presently available.

II. CONCEPTUAL LAYER

A database may be composed of one or more tables each of which has one or more columns, and one or more rows. For example: <tables id="tabl0003" num="0003"><table frame="all"><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center"><b>Name</b></entry><entry namest="col2" nameend="col2" align="center"><b>State</b></entry><entry namest="col3" nameend="col3" align="center"><b>Zip</b></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">John</entry><entry namest="col2" nameend="col2" align="left">VA</entry><entry namest="col3" nameend="col3" align="left">22204</entry></row><row><entry namest="col1" nameend="col1" align="left">Mary</entry><entry namest="col2" nameend="col2" align="left">DC</entry><entry namest="col3" nameend="col3" align="left">20013</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">Pat</entry><entry namest="col2" nameend="col2" align="left">MD</entry><entry namest="col3" nameend="col3" align="left">24312</entry></row></tbody></tgroup></table></tables>

In this small example, there is one table containing three columns, and three rows. The top row is the column names and is not considered a row in the database table. The term 'row' is interchangeable with the term 'record' also often used in database applications, and 'column' is interchangeable with the term 'field'. The primary distinction between the two sets of terms is that row and column are often used when the data is viewed in a list or spreadsheet table style view. and the terms field and record are used when the data is viewed one record at a time in a form style view.

A database may have more then one table. To this simple example, another table can be added called Purchases which lists purchases made by each Person. <tables id="tabl0004" num="0004"><table frame="all"><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center"><b>Name</b></entry><entry namest="col2" nameend="col2" align="center"><b>Product</b></entry><entry namest="col3" nameend="col3" align="center"><b>Quantity</b></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">John</entry><entry namest="col2" nameend="col2" align="left">apple</entry><entry namest="col3" nameend="col3" align="left">6</entry></row><row><entry namest="col1" nameend="col1" align="left">John</entry><entry namest="col2" nameend="col2" align="left">orange</entry><entry namest="col3" nameend="col3" align="left">4</entry></row><row><entry namest="col1" nameend="col1" align="left">Mary</entry><entry namest="col2" nameend="col2" align="left">kiwi</entry><entry namest="col3" nameend="col3" align="left">2</entry></row><row><entry namest="col1" nameend="col1" align="left">Pat</entry><entry namest="col2" nameend="col2" align="left">orange</entry><entry namest="col3" nameend="col3" align="left">12</entry></row><row><entry namest="col1" nameend="col1" align="left">Pat</entry><entry namest="col2" nameend="col2" align="left">kiwi</entry><entry namest="col3" nameend="col3" align="left">5</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">Pat</entry><entry namest="col2" nameend="col2" align="left">mango</entry><entry namest="col3" nameend="col3" align="left">10</entry></row></tbody></tgroup></table></tables>

Stored along with a database is some structure information about the tables contained within the database. This includes the name of the tables, if there are more then one, the names of the columns of the tables, and the structure of the data stored in the columns. In the above example, the first table is titled (for purposes of this example) "Person" and the second table "Purchases." In the Person table there are three columns: "Name", containing alphanumeric data; "State" containing two characters of alphanumeric data; and "Zip", which may be stored as five characters of alphanumeric data or as numeric data. In the Purchases table there are also three columns: "Name", containing alphanumeric data; Product, containing alphanumeric data; and Quantity, containing numeric data.

Also, stored along with the database are the primary keys for each of the tables. In most database systems each row must be uniquely identifiable. One or more columns together create the primary key which when the contents of those columns are combined uniquely identify each row in the table. In the Person table above, the column Name is unique in each row and Name could be the primary key column. However, in the Purchases table, "Name" does not uniquely identify each row since there are multiple Johns and Pats. In that table, both the "Name" and "Product" columns together uniquely identify each row and together form the primary key.

In the above example, there is an implied relationship between the two tables based on the common column title "Name". To determine how many oranges Virginians buy, a user could look in the Person table and find that John is the only Virginian and then go to the Purchases table to find that he bought four oranges. Some database managers explicitly store information about these relationships, including situations where the relationship is between two columns with different names.

The example above is very simple, and a user could readily understand what information the database held and how it was related. However, real world problems are not that simple. Though still rather simplistic compared to the complexity of many real world problems, the example database represented in <b>Figs. 1A-G</b> begins to show how difficult it might be for a user to understand what is contained in the database and how to draft a meaningful query. This is particularly difficult if the real meaning of the database is contrary to the naming conventions used when building it. For example, the Customer Table of <b>Fig. 1A</b> is not related directly to the Product Table of <b>Fig. 1B</b> even though they both have columns entitled NAME. However, they are related via the path CUSTOMER -&gt; ORDER -&gt; LINE_ITEM &lt;- PRODUCT (i.e. a customer has orders, an order has line_items, and each line_item has a product).

To shield the user from the complexity of the underlying database, a knowledgeable administrator may define a conceptual layer, which in addition to the basic database structure of table names and keys, and column names and types, also may include: foreign keys, name substitutions, table and column descriptions, hidden tables and columns, virtual tables, virtual columns, join path definitions and non-equijoins.

All of the forms of information that make up the conceptual layer can be stored alongside the database as delimited items in simple text files, in a database structure of their own, in a more compact compiled format, or other similar type of information storage. When a database is specified to be queried Query Assistant <b>1</b> has access to the basic structure of the database, which the database manager provides, to aid the user in formulating semantically correct queries. Optionally the user may choose to include the extended set of conceptual information which Query Assistant <b>1</b> can then use to provide a more intuitive query tool for the end user.

The conceptual layer information is stored internally in a set of symbol tables during operation. Query Assistant <b>10</b> uses this information to provide the user a set of choices conforming to the environment specified by the Administrator, and SQL Generator <b>20</b> uses the information, through a series of transformations, to generate the SQL query. .

A. Foreign keys

A table's foreign keys define how they relate to other tables. Two tables are joined by mapping the foreign key of one table to the primary key of the second table. A foreign key is defined by the columns within the first table that make up the foreign key, and the name of the second table which can join with the first table by matching its primary key with the first table's foreign key.

In the example above, the Purchases table with the foreign key "Name" can join the Person table with the primary key "Name." <tables id="tabl0005" num="0005"><img file="EP0803100B1_D0005.tif" /></tables>

If the two tables are joined based on their foreign and primary keys, the following new table is created: <tables id="tabl0006" num="0006"><table frame="all"><tgroup cols="5" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="31.50mm" /><colspec colnum="2" colname="col2" colwidth="31.50mm" /><colspec colnum="3" colname="col3" colwidth="31.50mm" /><colspec colnum="4" colname="col4" colwidth="31.50mm" /><colspec colnum="5" colname="col5" colwidth="31.50mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center"><b>Name</b></entry><entry namest="col2" nameend="col2" align="center"><b>State</b></entry><entry namest="col3" nameend="col3" align="center"><b>Zip</b></entry><entry namest="col4" nameend="col4" align="center"><b>Product</b></entry><entry namest="col5" nameend="col5" align="center"><b>Quantity</b></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">John</entry><entry namest="col2" nameend="col2" align="left">VA</entry><entry namest="col3" nameend="col3" align="left">22204</entry><entry namest="col4" nameend="col4" align="left">apple</entry><entry namest="col5" nameend="col5" align="left">6</entry></row><row><entry namest="col1" nameend="col1" align="left">John</entry><entry namest="col2" nameend="col2" align="left">VA</entry><entry namest="col3" nameend="col3" align="left">22204</entry><entry namest="col4" nameend="col4" align="left">orange</entry><entry namest="col5" nameend="col5" align="left">4</entry></row><row><entry namest="col1" nameend="col1" align="left">Mary</entry><entry namest="col2" nameend="col2" align="left">DC</entry><entry namest="col3" nameend="col3" align="left">20013</entry><entry namest="col4" nameend="col4" align="left">kiwi</entry><entry namest="col5" nameend="col5" align="left">2</entry></row><row><entry namest="col1" nameend="col1" align="left">Pat</entry><entry namest="col2" nameend="col2" align="left">DC</entry><entry namest="col3" nameend="col3" align="left">20013</entry><entry namest="col4" nameend="col4" align="left">orange</entry><entry namest="col5" nameend="col5" align="left">12</entry></row><row><entry namest="col1" nameend="col1" align="left">Pat</entry><entry namest="col2" nameend="col2" align="left">MD</entry><entry namest="col3" nameend="col3" align="left">24312</entry><entry namest="col4" nameend="col4" align="left">kiwi</entry><entry namest="col5" nameend="col5" align="left">5</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">Pat</entry><entry namest="col2" nameend="col2" align="left">MD</entry><entry namest="col3" nameend="col3" align="left">24312</entry><entry namest="col4" nameend="col4" align="left">mango</entry><entry namest="col5" nameend="col5" align="left">10</entry></row></tbody></tgroup></table></tables>

Rows from each of the two tables with the same value in their respective "Name" column were combined to create this new table. This is referred to as a One-to-Many relationship. For every one Person row there can be many Purchases rows. A relationship can also be One-to-One, which indicates that for every row in one table, there can be only one related row in another table. Both One-to-Many and One-to-One relationships may be optional or required. If optional. then there may not be a related row in a second table. In the illustrated embodiment, along with the foreign key in the conceptual layer an administrator may designate which of these four types of relationships (i.e. one-to-many, one-to-many optional, one-to-one, one-to-one optional) exists between the tables joined by the foreign key. In some database management systems, it is possible for a table to have multiple primary keys, in which case, the administrator must also designate to which primary key the foreign key is to be joined.

<b>Fig. 6</b> is a graphical representation of the relationships between the tables in <b>Figs. 1A-1G</b>. Each line represents a relationship between two tables and an arrow at the end of the line indicates a one-to-many optional relationship. The end with the arrow is the "many" end of the relationship. For example, between the SALESPEOPLE and CUSTOMERS tables there is a one-to-many relationship with multiple customers handled by each salesperson. Using the example tables of <b>Figs. 1A-G</b> and the relationships illustrated in <b>Fig. 6</b> a definition in the conceptual layer for the foreign keys would be: <tables id="tabl0007" num="0007"><table frame="all"><tgroup cols="5" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="31.50mm" /><colspec colnum="2" colname="col2" colwidth="31.50mm" /><colspec colnum="3" colname="col3" colwidth="31.50mm" /><colspec colnum="4" colname="col4" colwidth="31.50mm" /><colspec colnum="5" colname="col5" colwidth="31.50mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center"><b>Table</b></entry><entry namest="col2" nameend="col2" align="center"><b>Foreign Key</b></entry><entry namest="col3" nameend="col3" align="center"><b>Second Table</b></entry><entry namest="col4" nameend="col4" align="center"><b>Relationship Type</b></entry><entry namest="col5" nameend="col5" align="center"><b>Primary Key</b></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">CUSTOMERS</entry><entry namest="col2" nameend="col2" align="left">SALESPERSON#</entry><entry namest="col3" nameend="col3" align="left">SALESPEOPLE</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">LINE_ITEMS</entry><entry namest="col2" nameend="col2" align="left">ORDER#</entry><entry namest="col3" nameend="col3" align="left">ORDERS</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">LINE_ITEMS</entry><entry namest="col2" nameend="col2" align="left">PRODUCT#</entry><entry namest="col3" nameend="col3" align="left">PRODUCTS</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">ORDERS</entry><entry namest="col2" nameend="col2" align="left">CUSTOMER#</entry><entry namest="col3" nameend="col3" align="left">CUSTOMERS</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">ORDERS</entry><entry namest="col2" nameend="col2" align="left">SALESPERSON#</entry><entry namest="col3" nameend="col3" align="left">SALESPEOPLE</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">PRODUCTS</entry><entry namest="col2" nameend="col2" align="left">GROUP_ID</entry><entry namest="col3" nameend="col3" align="left">CODES</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">PRODUCTS</entry><entry namest="col2" nameend="col2" align="left">TYPE_ID</entry><entry namest="col3" nameend="col3" align="left">CODES</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">PRODUCTS</entry><entry namest="col2" nameend="col2" align="left">ALT-VENDOR#</entry><entry namest="col3" nameend="col3" align="left">VENDORS</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">PRODUCTS</entry><entry namest="col2" nameend="col2" align="left">VENDOR#</entry><entry namest="col3" nameend="col3" align="left">VENDORS</entry><entry namest="col4" nameend="col4" align="left">1-to-many opt.</entry><entry namest="col5" nameend="col5" align="left">1</entry></row></tbody></tgroup></table></tables>

The above chart represents the foreign key data which may be present in the conceptual layer. The chart is described by way of an example. According to the first row below the headings of the chart, there is a table CUSTOMERS with a foreign key defined by the column SALESPERSON#. This foreign key relates to the first primary key (note the 1 in the primary key column) of the SALESPEOPLE table by a one-to-many optional relationship. In other words, for every row in the SALESPEOPLE table there are zero or more related rows in the CUSTOMERS table according to SALESPERSON#.

The (2) next to the lines between CODES and PRODUCTS and VENDORS and PRODUCTS in <b>Fig. 6</b> indicates that there are actually two one-to-many relationships between those tables. This can be seen in the foreign key chart above. There are two sets of foreign keys linking PRODUCTS to VENDORS and PRODUCTS to CODES.

B. Name substitution

Name substitution is the process by which a table's or columns name as defined in the database structure is substituted with another more intuitive name for presenting to the user. This is particularly useful when dealing with a database management system which only provides limited naming capabilities (i.e. only one word). This process serves two primary purposes. First it allows an administrator to make the information available to the user in a given database more readily understandable, and second, it can be used to distinguish columns from different tables which have the same name, but are not related (i.e. column "Name" in the CUSTOMERS table (<b>Fig. 1A</b>) and column "Name" in the PRODUCTS table (<b>Fig. 1B</b>). In addition, it is possible to provide plural and singular names for tables.

For example, using the table in <b>Fig. 1A</b> it is possible to define the singular and plural names for the table as CUSTOMER and CUSTOMERS, and to rename the fields to provide more guidance to the user and distinguish conflicts as follows: <tables id="tabl0008" num="0008"><table frame="all"><tgroup cols="2" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="78.75mm" /><colspec colnum="2" colname="col2" colwidth="78.75mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center"><b>Column</b></entry><entry namest="col2" nameend="col2" align="center"><b>New Name</b></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">CUSTOMER#</entry><entry namest="col2" nameend="col2" align="left">Customer Number</entry></row><row><entry namest="col1" nameend="col1" align="left">NAME</entry><entry namest="col2" nameend="col2" align="left">Customer Name</entry></row><row><entry namest="col1" nameend="col1" align="left">CITY</entry><entry namest="col2" nameend="col2" align="left">Customer City</entry></row><row><entry namest="col1" nameend="col1" align="left">STATE</entry><entry namest="col2" nameend="col2" align="left">Customer State</entry></row><row><entry namest="col1" nameend="col1" align="left">ZIP_CODE</entry><entry namest="col2" nameend="col2" align="left">Customer Zip Code</entry></row><row><entry namest="col1" nameend="col1" align="left">SALESPERSON#</entry><entry namest="col2" nameend="col2" align="left">Salesperson Number</entry></row><row><entry namest="col1" nameend="col1" align="left">CREDIT_LIMIT</entry><entry namest="col2" nameend="col2" align="left">Credit Limit</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">BALANCE</entry><entry namest="col2" nameend="col2" align="left">Customer Balance</entry></row></tbody></tgroup></table></tables>

C. Table and column descriptions

Descriptions of the various tables and columns can be included in the conceptual layer to provide better understanding for the user. For example, The CUSTOMER table may have an associated description of "Records containing address and credit information about our customers." Then when the user highlights or otherwise selects the CUSTOMER table while building a query, the description will appear on a status line of the user interface or something similar. The same type of information can be stored for each of the columns which display when the columns are highlighted or otherwise selected for possible use in a query.

D. Hidden tables and columns

In the design of a database, it is often necessary to add columns that are important in relating the database tables but that are not used by the end user who will be forming queries on the database. For example, the SALESPERSON# column in the tables of Figs. <b>1A, 1D,</b> and <b>1E</b> are not important to the end user, who need only know that Paul Williams is the salesperson for American Butcher Block and Barn Door Furniture. The end user need not know that his internal number for use in easily relating the tables is 1. Accordingly, as part of the conceptual layer, an administrator can hide certain columns so that the user cannot attempt to display them or use them in formulating a query. When a column is hidden, it can still be used to join with another table. This same techniques can be used to prevent end users from displaying private or protected data, and to shield the user from the details of the database which might be confusing and unnecessary.

In some cases, there are tables which are used to link other tables together or are unimportant to the end user. Therefore, as part of the conceptual layer, an administrator can also hide certain tables so that the user cannot attempt to display them or use them in formulating a query. A hidden table, however, can still be used by the query system to perform the actual query -- it is just a layer of detail hidden from the end user. In addition, as described in more detail below, when virtual column and table techniques are used, columns may be included, for display to the end user, as elements of other tables. By hiding the original columns and/or tables, the administrator can, in effect, move a column from one table to another.

When designating elements that an end user can include in generating a semantically correct query, the Query Assistant will not designate the hidden tables and columns.

E. Virtual tables

Virtual tables are constructs that appear to the user as separate database tables. They are defined by the Administrator as a subset of an existing database table limited to rows that meet a specific condition. Initially, the virtual table has all the fields within the actual table upon which it is based, but it only contains a subset of the records. For example, the Administrator could define the virtual table BACKORDERS which includes all the records from the ORDERS table where the Status field contains the character "B". Then, when a user queries the BACKORDERS table, the user would only have access to those orders with backorder status.

The Administrator defines the virtual table according to a condition clause of the target language (in this case, SQL). In the above example, the table BACKORDERS would be defined as "ORDERS WHERE ORDERS.STATUS = 'B"'. In this way, the SQL generation portion of the virtual table is accomplished by a simple text replacement. Similarly, the condition could be stored in an internal representation equivalent to the SQL or other target language condition.

It is possible to define a virtual table without the condition clause. In that case, a duplicate of the table on which it is based is used. However, the Administrator can hide columns and add virtual columns to the virtual table to give it distinct characteristics from the table upon which it is based. For example, a single table could be split in two for use by the end user by creating a virtual table based on the original and then hiding half of the columns in the original table, and half of the columns in the virtual table.

F. Virtual columns

The conceptual layer may also contain definitions for virtual columns. Virtual columns are new columns which appear to the user to be actual columns of a table. Instead of containing data, the values they contain are computed when a query is executed. It is possible to add fields which perform calculations or which add columns from other tables. There are six primary uses for virtual columns: (1) <i>Moving/copying items from one table to another.</i> Often due to various database design factors, there are more tables in the physical database then in the user's conceptual model. In the example in <b>Figs. 1A-1G,</b> an end user might not consider orders as being multiple rows in multiple tables as is required with the LINE-ITEM, ORDER distinction of the example. The Administrator can specify in the conceptual layer that the user should see the field of LINE_ITEM (i.e. product, qty_ordered, qty_backordered, warehouse, etc.) as being part of the order table. Columns can be moved from one table to another with only one limitation that a primary key/foreign key relationship exist between the table the column is being moved from and the table the column is being moved to. These relationships are indicated in <b>Fig. 6</b> as the lines with the arrows. (2) <i>Creating a virtual column defined by a computation.</i> Virtual columns can be created by an administrator which are computations on existing columns in a table. For example, we could add a TURNAROUND column to the ORDERS table of Fig. 1F defined as "SHIP_DATE - ORDER_DATE". This would allow a user to easily create a query which asked to show what the turnaround time was for orders without having to actually specify the calculation in the query. (3) <i>Creating</i> a <i>virtual column defined using DBMS specific functions.</i> The target language of the Data Base Management System (DBMS) being used may have specific formatting or other data manipulation operations which could be used to present information to the user in a particular way. Even though the SQL Generator is designed to produce SQL, implementations of SQL differ from DBMS to DBMS. By the addition of a Lookup function, explicit joins can be defined in order to add columns from other tables or instances of the same table. The Lookup function can be used to define a virtual column and takes as parameters: a foreign key column (which is a foreign key column for the table where the virtual column is being placed, or a base table if it is a virtual table); and a reference column (which is a column in the table that the foreign key references). The remaining three uses employ this function to avoid complexities which are not addressed by current query systems. (4) <i>Eliminating complexity caused by alternate foreign keys.</i> Tables often have multiple ways of joining, represented by alternate foreign keys. This can be a source of confusion for the user. For example, using the tables of <b>Figs. 1A-1G</b>, the PRODUCT table (<b>Fig. 1B</b>) has two foreign keys, VENDOR# and ALT_VENDOR#. To aid the user in accessing the database, the Administrator would define virtual columns within the PRODUCT table for Vendor Name and Alternate Vendor Name, so that it appears to the user that they can easily find the vendor's names without resorting to looking in multiple tables for the information. However, this would generally confuse a query system because their are two foreign keys for use in joining the tables. By using the Lookup function for each of the Vendor Name and Alternate Vendor Name virtual columns, different foreign key joins can be specified for each of the columns, giving the user both the vendor and alternate vendor names. The definition of the virtual columns would be: <tables id="tabl0009" num="0009"><table frame="all"><tgroup cols="4" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="39.37mm" /><colspec colnum="2" colname="col2" colwidth="39.37mm" /><colspec colnum="3" colname="col3" colwidth="39.37mm" /><colspec colnum="4" colname="col4" colwidth="39.37mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center">Table</entry><entry namest="col2" nameend="col2" align="center">Virtual Column</entry><entry namest="col3" nameend="col3" align="center">Type</entry><entry namest="col4" nameend="col4" align="center">Definition</entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">PRODUCTS</entry><entry namest="col2" nameend="col2" align="left">VNAME</entry><entry namest="col3" nameend="col3" align="left">A</entry><entry namest="col4" nameend="col4" align="left">Lookup(PRODUCT.VENDOR#, VENDOR.NAME)</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">PRODUCTS</entry><entry namest="col2" nameend="col2" align="left">AVNAME</entry><entry namest="col3" nameend="col3" align="left">A</entry><entry namest="col4" nameend="col4" align="left">Lookup(PRODUCT.ALT_VENDOR#, VENDOR.NAME)</entry></row></tbody></tgroup></table></tables> (5) <i>Eliminating complexity caused by code tables</i> This is a special case of the alternate foreign keys case (4) above. Many databases have a code table whose purpose is to store the name and other information about each of several codes. The tables themselves only contain the code identifications. If a single table has multiple code columns which use the same table for information about the codes there is a potential for the alternate foreign key problem. The user instead of asking for products with "status_id = '007' and type_id = '002"' would prefer to ask for products with "status = 'open' and type = 'wholesale"'. Using the Lookup scheme, two virtual columns for the textual status and type can be added to the products table. (6) <i>Eliminating complexity caused by self-referencing tables</i> For example, each employee in an employee table may have a manager who himself is an employee -- the manager column refers back to the employee table. Using the Lookup function, virtual columns for each employees managers name, salary, etc. can be added to the employee table. To perform the actual query, a self join will be required. Using the employee table example for a table called EMP and a column MGR being a foreign key relating to the EMP table, the virtual column definitions would be: <tables id="tabl0010" num="0010"><table frame="all"><tgroup cols="4" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="39.37mm" /><colspec colnum="2" colname="col2" colwidth="39.37mm" /><colspec colnum="3" colname="col3" colwidth="39.37mm" /><colspec colnum="4" colname="col4" colwidth="39.37mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center">Table</entry><entry namest="col2" nameend="col2" align="center">Virtual Column</entry><entry namest="col3" nameend="col3" align="center">Type</entry><entry namest="col4" nameend="col4" align="center">Definition</entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">EMP</entry><entry namest="col2" nameend="col2" align="left">MNAME</entry><entry namest="col3" nameend="col3" align="left">A</entry><entry namest="col4" nameend="col4" align="left">Lookup(EMP.MGR, EMP.LNAME)</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">EMP</entry><entry namest="col2" nameend="col2" align="left">MSAL</entry><entry namest="col3" nameend="col3" align="left">N</entry><entry namest="col4" nameend="col4" align="left">Lookup(EMP.MGR, EMP.SAL)</entry></row></tbody></tgroup></table></tables>

G. Join path definitions

In certain circumstances it is possible to join two tables by multiple paths. For example, in the tables shown in <b>Figs. 1A-1G,</b> the SALESPEOPLE table can be joined with the ORDERS table by two different paths. This is easiest to see in <b>Fig. 6</b>. By following the direction of the arrow, SALESPEOPLE are connected directly to ORDERS or they can be connected to ORDERS via CUSTOMERS. In a query of the database, the manner in which the join is performed yields different results with different meanings. <ul id="ul0011" list-style="none" compact="compact"><li>(1) If SALESPEOPLE is joined directly with the ORDERS table, the result will indicate which salesperson actually processed the order.</li><li>(2) If SALESPEOPLE is joined to ORDERS via CUSTOMERS, the result will indicate the current salesperson for the customer on the order.</li></ul>

The Administrator can add to the conceptual layer a set of join paths for each pair of tables if desired. If multiple join paths are defined, a textual description of each join is also included. When the query is being generated, the system will prompt the user for which type of join the user prefers in an easy to understand manner. In the above example, when a user creates a query which joins the SALESPEOPLE and ORDERS tables the Query Assistant will generate the following dialog box: <tables id="tabl0011" num="0011"><table frame="all"><tgroup cols="2" colsep="1" rowsep="0"><colspec colnum="1" colname="col1" colwidth="78.75mm" /><colspec colnum="2" colname="col2" colwidth="78.75mm" /><tbody valign="top"><row><entry namest="col1" nameend="col2" align="left">Please clarify your query by indicating which of the following choices best characterizes the data you wish displayed:</entry></row><row><entry namest="col1" nameend="col1" align="left">1.</entry><entry namest="col2" nameend="col2" align="left">Use the salesperson that actually processed the order.</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">2.</entry><entry namest="col2" nameend="col2" align="left">Use the current salesperson for the customers on the order</entry></row></tbody></tgroup></table></tables> where the text in the choices is defined by the Administrator and correlates with the join path taken and used by the system.

If no join paths are defined for a given pair of tables, the shortest path is used by the system when creating a query. This can be determined by using a minimal spanning tree algorithm or similar techniques commonly known in the art.

H. Non-equijoins

The table joins discussed in the preceding examples have been equijoins. They are called equijoins because the two tables are combined or joined together based on the equality of the value in a column of each table (i.e. SALESPERSON# = SALESPERSON#, however, the column names need not be the same). In the illustrated embodiment, the Administrator can also provide in the conceptual layer definitions for non-equijoin relationships between tables which will join rows from two different tables when a particular condition is met. For example, another table ORDTYPE could be added to the example of <b>Fig. 1A-1G</b> that provides different classifications for orders of dollar amounts in different ranges: <tables id="tabl0012" num="0012"><table frame="all"><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center"><b>Low</b></entry><entry namest="col2" nameend="col2" align="center"><b>High</b></entry><entry namest="col3" nameend="col3" align="center"><b>Type</b></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">0</entry><entry namest="col2" nameend="col2" align="left">10000</entry><entry namest="col3" nameend="col3" align="left">Small</entry></row><row><entry namest="col1" nameend="col1" align="left">10000</entry><entry namest="col2" nameend="col2" align="left">50000</entry><entry namest="col3" nameend="col3" align="left">Medium</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">50000</entry><entry namest="col2" nameend="col2" align="left">1000000</entry><entry namest="col3" nameend="col3" align="left">Huge</entry></row></tbody></tgroup></table></tables>

Using a non-equijoin, a record from the ORDERS table could be joined with a record from the ORDTYPE table when ORDER_DOLLARS &lt;= LOW (from ORDTYPE table) &lt;= HIGH. Instead of an equality relationship, there is a relationship based on a range of values.

The Administrator codes the non-equijoin as an SQL condition. For the above example, the Administrator would specify that ORDERS should be joined with ORDTYPE "Where ORDERS.ORDER_DOLLARS &gt;= ORDTYPE.Low AND ORDERS,ORDER_DOLLARS &lt; ORDTYPE.High". It will also become evident that the procedures for specifying non-equijoins could be implemented in a manner similar to Query Assistant <b>10</b> to ensure correctness. The condition is stored in SQL so as to be directly used during the conversion from the intermediate language to the target language SQL. However, it is obvious to an artisan that the condition could be stored in an internal representation equivalent to the SQL condition or other target language.

III. QUERY ASSISTANT

Fig. 7 shows a block diagram of the Query Assistant <b>10</b>. It has two components: The Query Assistant User Interface (QAUI) <b>11</b> and the Query Assistant Expert System (QAES) <b>12</b>. QAUI <b>11</b> performs the functions of displaying the current state of the query to the user and providing a set of choice to the user for constructing a semantically correct query.

A. Query Assistant User Interface (QAUI)

QAUI <b>11</b> interacts with the user and QAES <b>12</b> to formulate a query in the intermediate language. Through the interface, the user initiates a query, formulates a query, runs a query, and views the results. <b>Fig. 8</b> shows the basic flow of control of QAUI <b>11</b>. When a user initiates a query at step <b>50</b>, QAUI 11 calls QAES <b>12</b> to initialize the blackboard at step <b>52</b>, then, in steps <b>54 - 58</b>, continuously presents to the user a set of choices based on the current context in the query, as limited by the rules in QAES <b>12</b>. After the user makes a selection at step <b>60,</b> the system QAUI <b>11</b> determines whether the user selected to clear the query (step <b>62</b>), and, if not, whether the user selected to run or cancel the query (step <b>66</b>), the blackboard is updated at step <b>68</b> and an intermediate language representation is updated at step <b>70.</b> This continues until the user either clears the query (at step <b>62</b>, in which case the intermediate language representation is cleared at step <b>64</b>) or cancels or runs the query (at step <b>66</b>), in which case the appropriate action is taken.

Fig. <b>9</b> shows the initial screen <b>110</b> presented by QAUI <b>11</b> to the user. Initial screen <b>110</b> has four areas: User Query window <b>112</b> (where the query in the intermediate language is built up); SQL Query window <b>114</b> (where the SQL equivalent of the User Query is displayed after the User Query is formulated); Result window <b>116</b> (where the result is displayed after the SQL Query is applied to the Database Management System); and menu bar <b>118</b> (providing access to a set of drop down menus that allow the user to select databases, select conceptual layers, interface to report generators, save and load queries, clear the current query, run the current query, set system defaults, etc.).

The user can invoke Query Assistant <b>10</b> by selecting it from a drop down menu under the Query heading. This brings up a query selection menu listing the various types of queries Query Assistant <b>10</b> can handle. This is based on the range of queries the intermediate language is capable of handling and the query generation routines built into Query Assistant <b>10</b>. Optionally, the administrator can limit the types of queries which the user can use on a given database by so specifying in the conceptual layer. If the user is limited to a single kind of query, then the query list is bypassed. In the illustrated embodiment, the query selection menu includes:<img file="EP0803100B1_D0006.tif" />

The <i>"Show..."</i> query covers approximately 99% of all queries and is the basic command to view certain columns or calculations thereon. The other queries are special types for percentage and comparison calculations. The type of result desired is obvious from the query excerpt in the display. This is in part due to the design of the intermediate language to make difficult query concepts easy to understand.

When the user selects the <i>"Show</i>..." query, the Create Show Clause dialog box <b>120</b> (shown in Fig. <b>10A</b>) is displayed. This is the primary means for interaction between the user and the Query Assistant. For purposes of illustration in the figures, items that can be selected by the user are in bold face, and items which cannot be selected are in italics. Other ways of distinguishing selectable items include: 'graying out' unselectable items by displaying them in lighter shades or different color; or inhibiting the display of nonselectable items so that only selectable items are displayed. The selections status (whether or not an item can be selected) is specified either by QAUI <b>11</b> or by a call to QAES <b>12</b>. Procedural rules are governed by the QAUI and expert system rules which define the selectable tables, columns, and operations are governed by QAES <b>12</b>. Procedural rules include, but are not limited to: <ul id="ul0012" list-style="none" compact="compact"><li>1. conditions or sort order on a query cannot be specified until something for display has been specified;</li><li>2. an individual column cannot be selected until it is highlighted;</li><li>3. a query cannot be run before something has been entered; and</li><li>4. items cannot be deleted until there is at least one item to delete.</li></ul>

The section designation window <b>121</b> of Create Show Clause dialog box <b>120</b> allows the user to designate what section or clause of the query is being entered. Window <b>121</b> includes Show, For, Sorted By, and With % of Total sections <b>121a-d</b>, respectively. The user need not designate the sections in any specific order except that at least one column must be designated to be shown before the other clauses.can be specified. However, the user may move back and forth between the sections. For example, a user may specify one column to show, then fill in For section <b>121b</b>, return to designate more columns to be shown, then designate a Sorted By column by selecting Sorted By section <b>121c</b>, etc.

The control section <b>122</b> of dialog box <b>120</b> includes a set of selection buttons <b>122a-d</b> by which the user can direct the system to run the query, clear the query, cancel creating a query, and create a column to show a computation. Computations are discussed in more detail below.

In item selection window <b>123,</b> the user can select tables and columns as specified in the conceptual layer, including any virtual tables or columns and any name substitutions. Any hidden tables or columns are hidden. Item selection window <b>123</b> includes table selection window <b>124</b>, column selection window <b>125</b>, description box <b>126</b>, and Select and Select All buttons <b>127a</b> and <b>127b.</b> For purposes of example, Fig. <b>10A</b> uses the tables of <b>Figs. 1A-1G</b> with several tables hidden, the columns renamed, and a generated column "THE COUNT OF CUSTOMERS" defined in the virtual layer as a Count on the table CUSTOMERS. By moving the highlighted bar from table to table in table selection window <b>124</b>, the list of available columns for the highlighted table is displayed in column selection window <b>125</b>. The Select and Select All buttons <b>127a, 127b</b> allow the user to select a column to show. Description box <b>126</b> shows a description for the highlighted table or column if a description is present in the conceptual layer.

The user can modify selected columns in the column modification window <b>128.</b> Columns selected for the Show clause are listed in display window <b>129</b>. The user can apply aggregate computations (i.e. count, total, average, minimum, and maximum) to the selected columns or unselect them via aggregate buttons <b>130a-h.</b>

After the user makes a selection (of a table in table selection window <b>124</b> or a column in column selection window <b>125</b>), QAUI <b>11</b> communicates with QAES <b>12</b> to update blackboard <b>13</b> and to request a new set of allowable selections. In addition, User Query window <b>112</b> of initial screen <b>110</b> is updated to reflect the query at that point in the intermediate language. If the selection made by the user causes certain items to become selectable or nonselectable, dialog box <b>120</b> is updated to reflect that. For example, <b>Fig. 10B</b> shows dialog box <b>120</b> after the user has selected the CUSTOMER BALANCE column of the CUSTOMERS table to display and has further selected to modify the column (indicated by the column being shown in display window <b>129</b>). In response, QAUI <b>11</b> has modified dialog box 120 in several ways. First, aggregate buttons <b>130a-h</b> are now selectable. QAES <b>12</b> has informed QAUI <b>11</b> that these buttons can be selected based on the determination that CUSTOMER BALANCE is numeric and that placing an aggregate on it would not create a semantically incorrect query. Had the user selected CUSTOMER NAME instead, QAES <b>12</b> would only have made Count button <b>130c</b> and None button <b>130h</b> selectable since the other types of aggregates require a numeric column. Also, For and Sorted By sections <b>121b, 121c</b>, in section designation window <b>121</b> are now selectable, as is the Run Query command <b>122a</b> in control section <b>122</b> since the Show section has something to show. User Query window <b>112</b> of initial screen <b>110</b> would now contain the string "SHOW CUSTOMER BALANCE".

<b>Fig. 10C</b> shows the state of dialog box <b>120</b> after the user has asked to find the average of CUSTOMER BALANCE (via Average button <b>130e</b>) and is preparing to select another column for display. Since the average aggregate has been placed on a numeric column, all the rows will be averaged together. Therefore, no joins to one-to-many tables are allowed and only other numeric columns which can be similarly aggregated can be selected. This has been determined by QAES <b>12</b> upon request by QAUI <b>11</b> and can be seen in dialog box <b>120</b> where all other tables and all non numeric columns have been made nonselectable. Had there been a virtual numeric column from another table present it also would not be selectable since a join is not allowed. If the user selects CREDIT LIMIT, QAUI <b>11</b> will be notified by QAES <b>12</b> that an aggregate is required and will put up a dialog box requesting which aggregate the user would like to use.

The user may also ask to see results that are actually computed from existing columns. In that case, the user can select Computation button <b>122d</b>. This selection causes QAUI <b>11</b> to display computation dialog box <b>135</b>, shown in <b>Fig. 10D</b>. Computation dialog box b allows the user to build computations of the columns. QAUI <b>11</b> requests of QAES <b>12</b> which tables, columns and operations are selectable here as well. The state of computation dialog box b as shown in <b>Fig. 10D</b> is as it would be at the start of a new query. However, all non-numeric fields are not selectable since computations must occur on numeric columns. This rule is stored in QAES <b>12</b>.

When the user selects Sort By section <b>121c</b> of section designation window <b>121,</b> QAUI <b>11</b> displays Sorted By dialog box <b>140</b>, shown in <b>Fig. 10E.</b> This dialog box is very similar to Create Show Clause dialog box <b>120</b>. As with the other dialog boxes, QAUI <b>11</b> works with QAES <b>12</b> to specify what columns can be selected by the user for use in the "Sort By ..."section. Note, the generated column THE COUNT OF CUSTOMERS is not selectable since it is actually an aggregate computation that cannot be used to sort the results of the query.

The For section, which is used to place a condition on the result, is more procedural in nature. If the user selects For section <b>121b</b> in section designation window <b>121</b>, QAUI <b>11</b> presents the user with a series of Create For Clause dialog boxes of the form shown in <b>Fig. 10F</b>, which provide a list of available choices in the creation of a <i>"For</i>..." clause. <b>Fig. 10F</b> shows the first Create For Clause dialog box <b>150</b>. A list of available choices is presented in choice window <b>151</b>. The displayed list changes as the user moves through the For clause. In dialog box <b>150</b>, the user can select to place a condition to limit the result of a query to rows "THAT HAVE" or "THAT HAVE NOT" the condition. When the user is required to enter a column in formulating the condition, the second Create For Clause dialog box <b>160,</b> shown in <b>Fig. 10G</b>, is displayed, with the tables, columns and operations designated as either selectable or nonselectable in a manner similar to the prior dialog boxes. In this way the user builds a condition clause from beginning to end.

The With percent of total section is simply a flag to add "WITH PERCENT OF TOTAL" to the end of the query. This provides a percent of total on a numeric field for every row in the result, if the query is sorted by some column.

The other three types of queries have similar sections which are handled by QAUI <b>11</b> in a similar way: <ul id="ul0013" list-style="none"><li>1. <i>What percent of... have ...</i> queries have three sections, the "What percent of ..." section, the "With..." section and the "Have ..." section. In the "What percent of ..." section the user is requested to select any table in the database as seen through the conceptual layer. Both the "With..." and "Have..." section ask for conditions as in the "For .." clause mentioned above.</li><li>2. <i>Compare... against...</i> queries have the following sections: "Compare..", "Against...", "Sort By...", and two "For... "sections. The query compares two numeric columns or computations which can have a condition placed on them in their respective "For ..." sections. Also the result can be sorted similarly to the "Show..." query. QAUI 11 handles each section similarly to the "Show..." , "For...", and "Sort By sections discussed above, with additional conditions placed on what can be selected set by QAES 12.</li><li>3. <i>Show... as a percentage of...</i> is treated the same by QAUI <b>11</b> as the Compare ... query above except that "Compare" is replaced with "Show". and "against" is replace with "as a percentage of'. This query is a special kind of comparison query.</li></ul>

The sections of the queries relate to the various portions of the target language SQL, however the actual terms such as "Show", "Compare", "That Have", etc. are a characteristic of the intermediate language used. As discussed more fully below, the intermediate language is designed in terms of the target language. Therefore QAUI <b>11</b> is designed with the specific intermediate language in mind in order to guide the user in creating only semantically correct queries.

B. Query Assistant Expert System (QAES)

QAES <b>12</b> is called by QAUI <b>11</b> to maintain the internal state information and to determine what are allowable user choices in creating a query. Referring to Fig. <b>7</b>, QAES <b>12</b> contains Blackboard <b>13</b> and Query Expert <b>14</b> which, based on the state of Blackboard <b>13</b>, informs QAUI <b>11</b> what the user can and cannot do in formulating a semantically correct query. Query Expert <b>14</b> provides QAUI <b>11</b> access to the blackboard and embodies the intelligence which indicates, given the current state of Blackboard <b>13</b>, what choices are available to the user for constructing a semantically correct query.

1. Blackboard

A blackboard is a conceptual form of data structure that represents a central place for posting and reading data. In the present invention, Blackboard <b>13</b> contains information about the current state of the system. As a query is being formulated by the user, Blackboard <b>13</b> is modified to reflect the selections made by the user.

Within the listed variables, Blackboard <b>13</b> maintains the following information: <ul id="ul0014" list-style="bullet" compact="compact"><li>whether or not a query is being created.</li><li>the type of query (Show, what % of, etc.)</li><li>the current clause (Show, For, Subquery, Sorted By, etc.)</li><li>the current set of choices of what can be selected by the user (for backup capability)</li><li>the set of tables selected for each of the current clause, whole query, and any subqueries</li><li>the table involved in a Count operation, if any (there can only be one)</li><li>the table involved in an aggregate operation (there can only be one)</li><li>the table involved in a computation operation (there can only be one)</li><li>the base table I virtual table relationship for any virtual table columns</li><li>a string defining each condition clause (i.e. For, With, Have)</li></ul>

To access and manipulate the data, the following routines are provided: <ul id="ul0015" list-style="bullet" compact="compact"><li>Initialize Blackboard (This sets all of the variable to zero or null state prior to the start of a query)</li><li>Set Query Type</li><li>Set Current Clause</li><li>Backup current set of selectable tables, columns, and operations.</li><li>Restore backup of set of selectable tables, columns, and operations.</li><li>Add table to set of tables selected for each of the current clause, whole query, and any subqueries</li><li>Remove table from set of tables selected for each of the current clause, whole query, and any subqueries</li><li>Read list of tables selected for each of the current clause, whole query, and any subqueries</li><li>Read/Write/Clear table involved in Count operation</li><li>Read/Write/Clear table involved in aggregate operation</li><li>Read/Write/Clear table involved in computation operation</li><li>Read/Write any base table &lt;-&gt; virtual table relationship for any virtual columns</li><li>Add/Remove text from string containing the whole intermediate language query and each condition clause (i.e. For, With, Have)</li></ul>

Possible methods for physical implementation of the blackboard include, but is not limited to, a set of encapsulated variables, database storage, or object storage, each with appropriate access routines.

After the user makes each selection in the process of building a query, Blackboard <b>13</b> is updated to reflect the current status of the query and Query Expert <b>14</b> can use the updated information to determine what choice the user should have next.

2. Query Expert

Query Expert <b>14</b> utilizes information stored on Blackboard <b>13</b> and information from the conceptual layer about the current database application to tell QAUI <b>11</b> what are the available tables, columns, and operations that the user can next select when building a query. Query Expert <b>14</b> makes this determination through the application of a set of rules to the current data. Although the rules used by the system are expressed in the illustrated embodiment by a set of If...Then... statements, it should be evident to the artisan that the rules may be implemented procedurally, through a forward or backward chaining expert system, by predicate logic, or similar expert system techniques.

Query Expert <b>14</b> examines each table and each column in each table to determine whether it can be selected by the user based on the current state of the query. Query Expert <b>14</b> also determines whether a selected table and/or column can be used in aggregate or computation operations. In addition, during the creation of a conditional "For ..." clause, Query Expert <b>14</b> addresses further considerations.

Query Expert <b>14</b> uses some similar and some different rules during construction of each of the sections of a query. For each section of the "Show ..." query described above, the rules employed by the Query Expert to designate what tables, columns, and operations the user can select in generating a semantically correct query are set forth below. <ul id="ul0016" list-style="none"><li>1. The <i>"Show</i>..." section. Below is a set of rules used to determine what tables, columns and operations are selectable by the user. The term "current clause" used within the rules refers to the entire <i>Show...</i> query. The current clause becomes important in the other three types of queries which have two separate sections used in comparisons. Each of those sections are separate clauses for the purpose of the rule base. <ul id="ul0017" list-style="none"><li><b>TABLES:</b> For each Table(x) in the database, if the following Table rules are all true, then the table is selectable. A rule of the form If.. Then TRUE has an implied Else FALSE at the end, and there is similarly an implied Else TRUE after an If.. Then FALSE. A table which is hidden, according to the conceptual layer, is not presented to the user for selection, but is processed by the rules in case virtual columns in non hidden tables are based on the hidden tables. If the hidden table cannot be selected, then any virtual table relying upon it cannot be selected. <ul id="ul0018" list-style="none"><li><b>Rule 211</b><dl id="dl0001" compact="compact"><dt>IF</dt><dd>the current clause is empty; Table(x) is a table already included in the current clause; there is only one other table in the current clause, and it can be joined with Table(x); or more then one table already exists in the current clause, and adding the new table results in a navigable set (There is a single common most detailed table between the tables.)</dd><dt>Then</dt><dd>TRUE</dd></dl></li><li><b>Rule 212</b><dl id="dl0002" compact="compact"><dt>IF</dt><dd>Table(x) is the base table of a virtual table with a condition clause; and the virtual table has already been selected</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 213</b><dl id="dl0003" compact="compact"><dt>IF</dt><dd>There is an aggregate being performed in the current clause; and Table(x) conflicts with the table that the aggregate is being applied to (If Table(x) is more detailed then the aggregate table or is joinable with Table(x) only through a more detailed table then there is a conflict)</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 214</b><dl id="dl0004" compact="compact"><dt>IF</dt><dd>Table(x) is following an aggregate command, and (there are no tables in the query; there is another aggregate present and either Table(x) already has an aggregate operation applied or has a one-to-one relationship with the already aggregated table; or there is not another aggregate present, and Table(x) is the most detailed table in the query (in one-to-many relationships, the many side is more detailed then the one side)</dd><dt>Then</dt><dd>TRUE</dd></dl></li></ul></li><li><b>COLUMNS:</b> For all Column(x) in a Table(x), if all the following Column rules are not false, the columns are selectable, else they are not. <ul id="ul0019" list-style="none"><li><b>Rule 221</b><dl id="dl0005" compact="compact"><dt>IF</dt><dd>Table(x) is not selectable</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 222</b><dl id="dl0006" compact="compact"><dt>IF</dt><dd>Column(x) is a virtual column; and the table on which the column is based is not selectable</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 223</b><dl id="dl0007" compact="compact"><dt>IF</dt><dd>There exists an aggregate on a Column(y); Column(y) is based on the same table as Column(x) or is based on a table with a one-to-one relationship with the table on which Column(x) is based; and Column(x) is non-numeric</dd><dt>Then</dt><dd>FALSE</dd></dl></li></ul></li><li><b>COMPUTATIONS:</b> The same rules apply to the use of tables and columns in computations accept for additional Computation rule 231, on Column(x) which must be true. <ul id="ul0020" list-style="none"><li><b>Rule 231</b><dl id="dl0008" compact="compact"><dt>IF</dt><dd>Column(x) is selectable; and Column(x) is numeric.</dd><dt>Then</dt><dd>TRUE</dd></dl></li></ul></li><li><b>AGGREGATE OPERATIONS:</b> For each aggregate operation Aggregate(x) (i.e., count, total, min, max, average) to be selectable for a selected column, Column(y), the following Aggregate rules must be true. <ul id="ul0021" list-style="none"><li><b>Rule 241</b><dl id="dl0009" compact="compact"><dt>IF</dt><dd>Column(y) is numeric; or Column(y) is non-numeric and Aggregate(x) = COUNT</dd><dt>Then</dt><dd>TRUE</dd></dl></li><li><b>Rule 242</b><dl id="dl0010" compact="compact"><dt>IF</dt><dd>There exists an aggregate on a Column(z); Column(z) is based on the same table as Column(y) or is based on a table with a one-to-one relationship with the table Column(y) is based on.</dd><dt>Then</dt><dd>TRUE</dd></dl></li><li><b>Rule 243</b><dl id="dl0011" compact="compact"><dt>IF</dt><dd>applying Aggregate(x) to Table(x) would cause a conflict with another table in the current clause (If Table(x) is less detailed then another table in the current clause then there is a conflict)</dd><dt>Then</dt><dd>FALSE</dd></dl></li></ul></li><li><b>SPECIAL RULE:</b> Special rules 251 is applied when a Column(x) is selected for display. Special rule 251 requires the user to enter an aggregate operation on the selected column. <ul id="ul0022" list-style="none"><li><b>Rule 251</b><dl id="dl0012" compact="compact"><dt>IF</dt><dd>There exists an aggregate on a Column(y); Column(y) is based on the same table as Column(x) or is based on table with a one-to-one relationship with the table Column(x) is based on.; Column(x) is selectable; and Column(x) is selected.</dd><dt>Then</dt><dd>Column(x) must have an aggregate applied, and the QAUI, at the direction of the QAES, will request one from the user.</dd></dl></li></ul></li></ul></li><li>2. The "Sort By ..." section. No computations or aggregates are allowed on the columns selected to sort the query by. Otherwise, the rules are similar to the "Show .." section, with some minor changes. Table rules <b>210</b> are the same, while Computation, Aggregate, and Special rules <b>231, 240</b>, and <b>251</b> are not applied. Finally, Column rule <b>223</b> is not applied since the "Sort By ..." section helps define a grouping order and will cause the aggregates to group by the Sort By columns. Therefore, even though an aggregate is already applied, the Sort By column cannot, and should not, be aggregated. It does not matter whether the column is numeric or non-numeric provided the table is selectable.</li><li>3. The "For ..." section is somewhat different from the previous two sections. Upon selecting a For condition, the user is led though a series of dialog boxes providing a set of choices to continue. Different rules apply at different points in the process of building a condition. Therefore, the knowledge as to what items are selectable by a user is contained in two forms. First, there is a procedural list of instructions, which direct the user in building a For clause, and second, there is a set of rules that are applied at specific times to tables, columns, and operations in a similar manner to the Show and Sort By clauses. Pseudocode representations of the procedural knowledge used in directing a user to create a For clause is shown below. Although not explicitly stated in the pseudocode, after every selection made by the user, Blackboard <b>13</b> is updated, and the current query is updated for display to the user. Also, at any time during the creation of the For clause, the user may: clear the query (which will clear the blackboard and start over); backup a step (which will undo the last choice made); or run the query (if at an appropriate point). The pseudocode is shown to cover a certain set of condition clause types, however, it will be apparent to the artisan how additional condition types can readily be added. According to the pseudocode, QAUI <b>11</b> calls the <b>For_Clause_Top_Level_Control</b> procedure to initiate the creation of a For clause. The <b>Choose_entity</b> function below is used to select a table or column and is described in more detail below. Procedure <b>For_Clause_Top_Level_Control 310</b> is a loop that creates conjunctive portions of the For clause until there are no more ANDs or ORs to be processed.<img file="EP0803100B1_D0007.tif" /><b>Function Make_For_Clause 320</b> handles some special types of For clauses by itself and sends the general type of For clause constructs (i.e. constraints) to function Make_Constraint <b>330</b>. The result of the function is an indication as to whether there is a pending And or Or in the query.<img file="EP0803100B1_D0008.tif" /><img file="EP0803100B1_D0009.tif" /><b>Function Make_Constraint 330</b> is the heart of the creation of the "For" clause. It takes as parameters some QAES rule parameters and a flag which indicates whether every part of the For clause is already present when this function is called. The result of the function is an indication whether or not the last thing selected was an AND or Or.<img file="EP0803100B1_D0010.tif" /><img file="EP0803100B1_D0011.tif" /><img file="EP0803100B1_D0012.tif" /> In the above pseudocode, the function <b>Choose_Entity</b> is not defined. This function uses the second type of knowledge. The function is called with a set of parameters, and based on those parameters, the user is asked to choose either a table or column. As with the other clauses, this information is presented to the user in a manner to distinguish which choices the user can make. QAES 12 determines what selection the user may make by applying a set of rules to the tables and columns as in the other clauses. There is an additional element, however, in the rule base for the For clause. The rule base is expanded to include special circumstances which are specified by a parameter <b>Choose_Entity.</b> The parameter, in the pseudocode, takes the form of a list of conditions in a string separated by commas. There are two types of condition, those which inform QAES <b>12</b> what type of dialog item the user will be selecting and therefore what type of dialog box to display, and those which are conditions in the rules. Types of item parameters include: <ul id="ul0023" list-style="none" compact="compact"><li>Entity - indicates that the user needs to select a table;</li><li>Attribute - indicates that the user needs to select a column; and</li><li>Numeric Attribute - indicates that the user can only select a numeric column.</li></ul> If the user needs to select a table, then the rules will not be applied to the columns, since they will not be displayed. The condition type parameters are: <ul id="ul0024" list-style="none" compact="compact"><li>NMD - The table can be No More Detailed then any other table in the current clause</li><li>NLD - The table can be No Less Detailed Then any other table in the current clause</li><li>Detail - The table must be the most detailed table in the current clause</li><li>ForTable - The table must be Identical or one-to-one with any tables in the For clause</li><li>OFENT - The table must be Identical or one-to-one with any table in current clause</li><li>NOTENT - The table must not be identical or one-to-one with any table in current clause.</li></ul> In a one-to-many relationship, the table on the many side of the relationship is more detailed then the table on the one side. As discussed previously, the term "current clause" actually refers to the entire query in a Show ... query. The current clause becomes important in the other three types of queries which have two separate section used in comparisons. Each of those sections are separate clauses for the purpose of the rule base. The rule base used in the For clause is set out below. <b>TABLES:</b> Table rules <b>211-214</b> are applied. In addition, the following Table Parameter rules are applied. <ul id="ul0025" list-style="none"><li><b>Rule 311</b><dl id="dl0013" compact="compact"><dt>IF</dt><dd>Parm contain "NMD"; and a less detailed table then Table(x) is already in the current clause</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 312</b><dl id="dl0014" compact="compact"><dt>IF</dt><dd>Parm contains "NLD"; and a more detailed table then Table(x) is already in the current clause</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 313</b><dl id="dl0015" compact="compact"><dt>IF</dt><dd>Parm contains "Detail"; and (there is a more detailed table then Table(x) in the current clause; Table(x) already exists in the current clause; or a table with a one-to-one relationship with Table(x) exists in the current clause and it is aggregated)</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 314</b><dl id="dl0016" compact="compact"><dt>IF</dt><dd>Parm contains "For Table"; Table(x) doesn't exist in the For clause; and Table(x) does not have a one-to-one relationship with any table in the For clause</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 315</b><dl id="dl0017" compact="compact"><dt>IF</dt><dd>Parm contains "OFENT"; Table(x) doesn't exist in the current clause; and Table(x) does not have a one-to-one relationship with any table in the current clause</dd><dt>Then</dt><dd>FALSE</dd></dl></li><li><b>Rule 316</b><dl id="dl0018" compact="compact"><dt>IF</dt><dd>Parm contains "NOTENT"; and (Table(x) exists in the current clause; or Table(x) has a one-to-one relationship with any table in the current clause)</dd><dt>Then</dt><dd>FALSE</dd></dl></li></ul><b>COLUMNS:</b> In addition to Column rules <b>221-223</b>, the following Column Parameter rule is applied. <ul id="ul0026" list-style="none"><li><b>Rule 321</b><dl id="dl0019" compact="compact"><dt>IF</dt><dd>Parm contains "Numeric Attribute"; Column(x) is non-numeric</dd><dt>Then</dt><dd>FALSE</dd></dl></li></ul> Computation rule <b>231</b> is also applied. </li><li>4. "With percent of Total" check box. The user can add this phrase to the end of a query if two things are true: (1) the last item in the Show clause was numeric; and (2) there is a sort specified in the Sort By clause.</li></ul>

The same set of rules are used in the equivalent sections of the other three query types as discussed in the earlier section on QAUI <b>11</b>. These queries are considered two clause queries with each clause represented by the ellipses in the queries. Blackboard <b>13</b> is set to the current clause, and the rules which refer to current clauses use the clause being built by the user. The rules are primarily the same as the Show ... query with the following caveats: <ul id="ul0027" list-style="none"><li>1. In the <i>What percent of... have ...</i> queries, the "What percent of ..." section is limited to one table in the database, so there are no real rules applied. The "With..." and "Have..." sections are then the same as the "For..." section in the Show ... query.</li><li>2. In the <i>Compare... against...</i> queries, the "Compare .." and "Against ..." sections are limited to a numeric columns, including aggregates and computations. Also in each of the sections, there can be only one column, aggregate or computation. The "For..." and "Sort By ..." clause use the same rule sets as those in the Show ... queries.</li><li>3. In the <i>Show... as a percentage of...</i> queries, the "Show .." and "as a percentage of ..." sections are limited to a numeric columns, including aggregates and computations. Also in each of the sections, there can be only one column, aggregate or computation. The "For..." and "Sort By ..." clause use the same rule sets as those in the Show ... queries.</li></ul>

To illustrate how Query Assistant <b>10</b> prevents the user from having the opportunity to formulate the incorrect query involving the three tables CUSTOMERS, ORDERS and LINE_ITEMS from <b>Figs. 1A, 1F,</b> and <b>1G,</b> respectively, with the relations shown in <b>Fig. 6</b>. the steps that the user would take to attempt the incorrect query are described. First, the user would invoke Query Assistant <b>10</b> and select a Show ... query. At this point all of the tables and columns would be selectable since nothing has yet been selected, however, the Run Query box is not selectable, and the other sections of the query are not selectable until there is something in the Show section. Next, the user would select the column Name in the CUSTOMERS table for display. Again, after applying the rules, all of the tables and columns are selectable. This is indicated by the rule base because all tables can be joined with CUSTOMERS, and there has not been an aggregate defined. Next, the user would select Order Dollars from the ORDERS table to display. All tables and columns are still selectable for the same reason.

Next, the user would select to modify Order Dollars. After applying the rules, QAES 12 would indicate that any of the aggregates can be applied to Order Dollars since Order Dollars is numeric and there are no other aggregates. Next, the user would select a Total on Order Dollars. After applying the rules, QAES <b>12</b> would determine that the LINE_ITEMS, PRODUCT, CODE, and VENDOR tables are no longer selectable because of Table rule <b>213</b>. Also, only numeric columns are selectable in the ORDERS table and they must be aggregated as dictated by Column rule <b>223</b> and Special rule <b>251</b>.

Finally, columns in the CUSTOMERS table are selectable, but cannot be aggregated because of Aggregate rule <b>242</b>. The user is not allowed to select the LINE_ITEMS table once an aggregate is placed on ORDER DOLLARS so the incorrect query cannot be formulated. Similarly, if a column in the LINE_ITEMS table had been selected prior to placing the Total on Order Dollars, Order Dollars could not be aggregated because of Aggregate rule <b>243</b>.

More complex queries are handled in the same way. After each user selection, QAES <b>12</b>, through QAUI <b>11</b>, provides a list of available choices based on knowledge it has about databases and the information it contains in the conceptual layer. It applies rules to all applicable tables, columns, and operations to determine the list of selectable items. During the creation of a For clause, a procedural component is introduced, but the method of operation is substantially the same.

IV. INTERMEDIATE LANGUAGE

As discussed earlier, because the set of semantically valid queries cannot be described by a context-free grammar, a grammar is not given for the intermediate language, and one is not used to create user choices. Rather, the intermediate language is defined by the templates and choices presented to the user by Query Assistant <b>10</b>. The templates are screen definitions whose contents (i.e. picklist contents and active vs. inactive elements) are governed by QAES <b>12</b>. Condition clause generation is driven by a separate module in QAES <b>12</b>. The definition of the intermediate language is precisely those queries that can be generated bv Query Assistant <b>10.</b>

The design of the intermediate language, however, is driven from the bottom (SQL Generator <b>20</b>) and not the top (Query Assistant <b>10</b>). The architecture of Query System <b>1</b> is designed to minimize the amount of processing by SQL Generator <b>20</b> by making the intermediate language as similar as possible to the target language (SQL) while providing a more easily understandable set of linguistic constructs. Building upon the design of the language, Query Assistant 10 is built to implement the production of semantically correct queries using the linguistic constructs of the language, which in turn can further simplify the design of SQL Generator <b>20</b>.

In natural language systems, the problem lies in converting a representation of a natural language to a target language, such as SQL. In the present invention, conversion of the intermediate language to a target language is straightforward because the intermediate language was designed to accommodate this process. The intermediate language is designed by starting with the target language (in the illustrated embodiment, SQL), and making several modifications.

First, the grouping and table specification constructs, which in SQL are specified by the GROUP BY, HAVING, and FROM clauses respectively, are deleted, so that the user need not specify them. Rather, this information can be inferred readily from the query. For example, if the user selects a column for display, the table from which the column comes needs to be included in the table specification (i.e. FROM clause). When a user selects to view columns in a table without selecting a primary key, the user would likely want to see the column results grouped together, so that like values are in adjacent rows of the output and duplicates are discarded. This is specified in the GROUP BY clause of SQL, but it can be inferred. In SQL, the HAVING clause is a special clause which operates as a WHERE clause for GROUP BY items. This is also readily inferred from which columns are grouping columns and if they have specified conditions.

Second, join specifications are deleted from the condition clause. SQL requires an explicit definition of the joins in its WHERE clause (i.e. WHERE CUSTOMER.CUSTOMER# = ORDER.CUSTOMER#). This information can be inferred or specifically requested when creating a query if necessary, but it does not form a part of the intermediate language query.

Third, specific and readily understandable patterns are defined for each type of subquery supported by the intermediate language. For example, the English pattern "MORE THAN 2 &lt;category&gt;" can be defined to have a specific SQL subquery expansion.

Fourth. the remainder of the target language is replaced with easily understandable words or phrases that, when strung together, form a comprehensible sentence. For example, using SQL, "SELECT' is replaced with "Show", "WHERE" is replaced with "For", "ORDER BY" is replaced with "Sorted By" and so on.

Finally, synonyms are provided for various words, phrases and constructs. Target language constructs may look differently in the intermediate language depending on the type of query to be formed if the query is to be an easily understood sentence. This also allows the user multiple ways of specifying concepts, including, but not limited to: dates (i.e. Jan. 1, 1994 v. 01/01/94, etc.), ranges ( between x and y, &gt; x and &lt; y, last month, etc.); and constraints (&gt;, Greater Then, Not Less Than or Equal).

V. SQL GENERATOR

Given the design of the intermediate language described above, SQL Generator <b>20</b> need only perform two basic functions. First, it needs to infer the implicit portions of the query that are explicitly required in the target language, such as the GROUP BY clause in SQL, or the explicit joins in the WHERE clause. This information is easily inferred because of the design of the intermediate language. Second, SQL Generator <b>20</b> needs to resolve synonyms and transform the more easily understood intermediate language into the more formal target language through a series of transformations by pattern substitution. It is this set of patterns that give the intermediate language its basic look and feel.

Internally, the intermediate language has a component that is independent of the database application, and a component that is specific to the database application. The application independent component is represented by the sets of patterns used in the pattern substitution transformations and the set of routines used to infer implicit information. The application specific component is represented by the conceptual layer which contains information used in both basic functions of SQL Generator <b>20</b>.

SQL Generator <b>20</b> has no expertise concerning what is and is not a semantically valid query in the intermediate language. If the user bypasses Query Assistant 10 to directly enter a query using the syntax of the intermediate language, the user can obtain the same incorrect results that can be obtained with conventional point-and-shoot query tools.

A. Flow of Control

<b>Figs. 11A</b> and <b>11B</b> depict a flowchart of the flow of control of SQL Generator <b>20</b>. Each of the steps is described in detail below. SQL Generator <b>20</b> applies a series of transformations to the intermediate language input to produce a resulting SQL statement. If one of these transformations fails, an error message will be displayed. An error message is only possible if the input query is not part of the intermediate language (i.e. was not, and could not be, generated by Query Assistant <b>10</b>). SQL Generator <b>20</b> takes as input an intermediate language query and produces an SQL statement as output by applying the steps described below.

In step <b>402</b>, the intermediate language query is tokenized, i.e. converted to a list of structures. Each structure holds information about one token in the query. A token is usually a word, but can be a phrase. In this step, punctuation is eliminated, anything inside quotes is converted to a string, and an initial attempt is made to categorize each token.

In steps <b>404</b> and <b>406</b>, the intermediate language query is converted to an internal lexical format. The conversion is done via successive pattern matching transformations. The clause is compared against a set of patterns until there are no more matched patterns. The lexical conversion pattern matcher is discussed in more detail below. The resulting internal format differs from the intermediate language format in several ways: <ul id="ul0028" list-style="none"><li>(a) Synonyms within the intermediate language for the same construct are resolved to a single consistent construct (i.e., "HAS" and "HAVE" become "HAVE").</li><li>(b) Synonyms for tables and columns are resolved, utilizing the names specified in the conceptual layer and by converting column names to fully qualified column names in the form of Table_Name.Column_Name</li><li>(c) Dates are converted to a Julian date format</li><li>(d) Extraneous commas and ANDs (except those in condition clauses) are deleted</li><li>(e) Condition clauses are transformed to match one or more predefined WHERE clause patterns stored as ASCII text in an external file</li><li>(f) Special symbol are inserted to demarcate the beginning and middle of "what percent of ...", "show... as a % of...", and "compare..." queries.</li><li>(g) A special symbol is inserted to demarcate the object of every FOR clause.</li><li>(h) Certain words designated as Ignore words are eliminated. (i.e. The, That, etc.)</li></ul>

When no further patterns can be matched, control transfers to step <b>408</b>, where it is determined whether CREATE VIEW statements are necessary. If so, in steps <b>410</b> and <b>412</b>, SQL Generator <b>20</b> is called recursively as a subroutine to generate the required views. As the view is generated the recursive call to SQL Generator <b>20</b> is terminated. CREATE VIEW (an SQL construct) is required for queries in the intermediate language which call for percentage calculations or otherwise require two separate passes of the database (i.e. comparisons). The types of queries that Query Assistant <b>10</b> can produce that require a CREATE VIEW statement are of a predetermined finite set, and SQL Generator <b>20</b> includes the types of queries which require CREATE VIEW generation. An example type of query where it is required is "Compare X against Y" where X and Y are independent queries that produce numeric values. Within each recursive call to SQL Generator <b>20</b>, pattern matching is conducted to resolve newly introduced items. Control then passes to step <b>414</b>.

In step <b>414</b>, the internal lexical format is converted into an internal SQL format, which is a set of data structures that more closely parallel the SQL language. The internal SQL format partitions the query into a sets of strings conforming to the various SQL statement elements including: SELECT columns, WHERE clauses, ORDER BY columns, GROUP columns, Having clause flag, FROM table/alias pairs. JOINs. In this step, the SELECT, and ORDER BY sections are populated, but WHERE clauses are maintained in lexical format for processing at the next step. The other elements are set in the following steps if necessary.

In the ensuing steps <b>416</b> and <b>418</b>, the lexical WHERE phrases are compared with a set of patterns stored in an external ASCII text file. If a match is found, a substitution pattern found in the external file is used for representing the structure in an internal SQL format. In this way, the WHERE clause is transformed from the intermediate language to the internal SQL format equivalent.

If any table references have been introduced into the internal SQL structure as columns, they are converted to column references in step <b>420</b>. This can occur on queries like "show customers". Virtual table references are also expanded in this step using the conceptual layer information to include the table name and the virtual table condition, if present, which is added to the internal structure.

If there are any columns in the ORDER BY clause that are not in the SELECT, they are added to the SELECT in step <b>422.</b> In step <b>424</b>, Julian dates are converted to dates specified in appropriate SQL syntax. Next, in step <b>426</b>, virtual columns are expanded into the expressions defined in the conceptual layer by textual substitution. This is why virtual column expressions are defined according to SQL expressions or other expressions understood by the DBMS. In this step, the expression of a virtual column will be added to the WHERE clause -- a Lookup command will simply make another join condition in the WHERE clause.

In step <b>428</b>, the FROM clause of the SQL statement is created by assigning aliases for each table in the SELECT and WHERE clauses, but ignoring subqueries that are defined during the pattern matching of steps <b>416</b> and <b>418</b>. In step <b>430</b>, the ORDER BY clause is converted from column names to positional references. Some SQL implementations will not accept column names in the ORDER BY clause -- they require the column's position in the SELECT clause (i.e. 1, 2 etc.). This step replaces the column names in the ORDER BY clauses with their respective column order numbers.

In step <b>432</b>, the navigation path is computed for required joins. This is done using a minimal spanning tree as described above. This is a technique commonly used for finding the shortest join path between two tables, but other techniques will work equally well. If additional tables are required then they are added. Also, by default, the shortest join path is created. However, if the user designated a different join path which was predefined by the administrator and put in the conceptual layer, that path is used. If it is determined in step <b>434</b> that new tables are required, they are added in step <b>436</b> to the FROM clause. Then, in step <b>438</b>, the WHERE clause join statements are created in the internal SQL structure.

In step <b>440</b>, SELECT is converted to SELECT DISTINCT, if necessary. This is required if the query does not include the primary key in the Show clause of the query and there are only non-numeric columns (i.e. "Show Customer State and Customer City"). The primary keys are defined as Customer Number and Customer Name in the CUSTOMERS table. SELECT DISTINCT will limit the query to distinct sets and will group the results by columns in the order listed. Using SELECT alone will result in one output line for every row in the table.

In step <b>442,</b> the GROUP BY clause is added to internal SQL (if necessary), as are any inferred SUMs. This is required if the query does not include the primary key in the Show clause of the query and there are numeric columns (i.e. "Show Customer State and Customer Balance"). The primary key is the Customer Number in the CUSTOMERS table and here Customer Balance is a numeric field. What the user wants is to see the balance by state. Without including Group By and SUM, there will be a resulting line for every row in the CUSTOMERS table. This step places any numeric fields in SUM expression and places all non-numeric fields in a GROUP BY clause. For example, the above query would produce the following SQL.<img file="EP0803100B1_D0013.tif" />

In step <b>444</b>, COUNTs are converted to COUNT (*), if necessary. This is required, as an SQL convention, where the user requests a count on a table. For example, the query "Show The Count of Customers" produces the SQL code<img file="EP0803100B1_D0014.tif" />

Finally, in step <b>446</b>, the internal SQL format is converted into textual SQL by passing it through a simple parser.

B. Pattern Matching

Steps <b>404</b> and <b>416</b> transform the intermediate language query using pattern matching and substitution techniques. These steps help to define the intermediate language more then any other steps. By modifying these pattern/substitution pairs the intermediate language could take on a different look and feel using different phrases. Accordingly, Query Assistant <b>10</b> would need to be able to produce those phrases. Further, by adding patterns, the user can be given more ways of entering similar concepts (when not using Query Assistant <b>10</b>), and more types of subqueries can be defined. For every new type of subquery defined as a pattern. the Where clause generation function of Query Assistant <b>10</b> would need to be modified to provide the capability.

Two types of patterns used in SQL Generator <b>20</b>. The first, used in step <b>404</b>, is a simple substitution, while the second, used in converting Where clauses in step <b>416</b>, is more complex because it can introduce new constructs and subqueries.

1. Lexical conversion pattern matching

In the lexical conversion pattern matching of step <b>404</b>, a text string of the query is compared to a pattern, and if a substring of the query matches a pattern, the substring is replaced with the associated substitution string. Patterns take the form of:    <i>PRIORITY SUBSTITUTION</i> &lt;- <i>PATTERN</i> PRIORITY is a priority order for the patterns which takes the form of #PRIORITY-? with ? being a whole number greater then or equal to 0. This provides an order for the pattern matcher with all #PRIORITY-0 patterns being processed before #PRIORITY-1 patterns and so on. Within a given priority, the patterns are applied in order listed. If the pattern does not begin with a priority, it has the same priority as the most recently read pattern (i.e. all patterns have the same priority until the next priority designation)

PATTERN is what is compared against the query to find a match. Textual elements which match directly with words in the query along with the following key symbols: <dl id="dl0020" compact="compact"><dt>{}</dt><dd>or</dd><dt>[]</dt><dd>a single phrase</dd><dt>~</dt><dd>optional</dd><dt>!???x</dt><dd>variable that matches anything, x is a number.</dd><dt>!ENTx</dt><dd>table variable which matches any table name, x is a number in case of multiple tables in the pattern.</dd><dt>!ATTx</dt><dd>column variable which matches any column name, x is a number in case of multiple columns in the pattern.</dd><dt>!VALx</dt><dd>value variable which matches any numeric value, x is a number for multiple values in the pattern.</dd><dt>!FUNCTIONx</dt><dd>function variable which matches a function that can be applied to a column (i.e. SUM, AVG, etc.), x is a number for multiple functions in the pattern.</dd></dl>

SUBSTITUTION is the replacement text. Every instance of !???, !ENTx. !ATTx, or !VALx is replaced with the table, column or value bound to the variable in the PATTERN.

As an example, the pattern "PRIORITY-0 AND THAT HAVE &lt;- {[AND HAVE ] [ AND HAS]}" indicates that the phrases "AND HAVE" and "AND HAS" are synonyms for the phrase "AND THAT HAVE" and will be accordingly substituted. The brackets signify phrases. The braces signify multiple synonyms. The "#PRIORITY-0" entries define the pattern as having a priority of 0 so that this rule would apply before any priority 1 rules, etc.

Another example pattern is "(!ATT1 &gt;= !???1 and !ATT1 &lt;= !???2 ) &lt;- {[ !ATT1 BETWEEN !???1 AND !???2] [!ATT1 FROM !???1 TO !???2]}". In this case the pattern would match substrings of the form of "BALANCE BETWEEN 10000 AND 50000" or "BALANCE FROM 10000 TO 50000" and would substitute it with "BALANCE &gt;= 10000 AND BALANCE &lt;= 50000". As is evident from the form of the patterns, the intermediate language which is understandable to the SQL Generator can be simply varied by changing these patterns or adding new patterns to recognize different words or phrases.

The set of patterns used in step 404 of the illustrated embodiment (i.e., for one instance of an intermediate language) is shown below. <ul id="ul0029" list-style="none"><li><b>Pattern 501</b> #PRIORITY-0 AND THAT HAVE &lt;- {[ AND HAVE ] [ AND HAS ]}</li><li><b>Pattern 502</b> #PRIORITY-0 AND THAT DO NOT HAVE &lt;- {[AND DO NOT HAVE] [ AND DOES NOT HAVE ]}</li><li><b>Pattern 503</b> #PRIORITY-0 WHERE NOT &lt;- {WITHOUT [THAT DO NOT HAVE ] [ THAT DOES NOT HAVE ]}</li><li><b>Pattern 504</b> #PRIORITY-0 HAVE NOT &lt;- {[ DO NOT HAVE ] [ DOES NOT HAVE]}</li><li><b>Pattern 505</b> #PRIORITY-2 !ENT1, !ENT2 &lt;- [!ENT1 !ENT2]</li><li><b>Pattern 506</b> #PRIORITY-5 !ENT1, !ATT1 &lt;- [!ENT1 !ATT1]</li><li><b>Pattern 507</b> #PRIORITY-0 PCT_TOTAL &lt;- [WITH {% PERCENT PERCENTAGE} OF TOTAL ]</li><li><b>Pattern 508</b> #PRIORITY-0 !ATT1 PCT_TOTAL &lt;- [ !ATT1 WITH {% PERCENT PERCENTAGE} OF TOTAL SUBTOTAL ]</li><li><b>Pattern 509</b> #PRIORITY-0 WHAT_PERCENT_BEGIN &lt;- [WHAT {% PERCENT PERCENTAGE} ∼ OF ]</li><li><b>Pattern 510</b> #PRIORITY-0 AS_PCT_MIDDLE &lt;- [AS A {% PERCENT PERCENTAGE} OF ]</li><li><b>Pattern 511</b> #PRIORITY-0 COMPARE_BEGIN &lt;- COMPARE</li><li><b>Pattern 512</b> #PRIORITY-2 OFENTITY!! !ENT1 WHERE &lt;-    [FOR{[!ENT1 THAT HAVE ]    [!ENT1 THAT HAS ]    [!ENT1 CANTFOLLOW XDATE]}]</li><li><b>Pattern 513</b> #PRIORITY-2 OFENTITY!! !ENT1 WHERE &lt;- {[WHERE !ENT1 HAVE] [WHERE !ENT1 HAS]}</li><li><b>Pattern 514</b> #PRIORITY-2 !ATT1 = !VAL1 &lt;- [!ATT1 = " !VAL1"]</li><li><b>Pattern 515</b> #PRIORITY-2 XDATE MTH !MTH1 ENDPT &lt;- !MTH1</li><li><b>Pattern 516</b> #PRIORITY-2 XDATE MTH !VAL1 DAY !VAL2 CYR !VAL3 ENDPT &lt;- [!VAL1 / !VAL2 / !VAL3]</li><li><b>Pattern 517</b> #PRIORITY-2 XDATE MTH !MTH1 DAY !VAL1 CYR !VAL2 ENDPT &lt;- [!MTH1 !VAL1 ∼ !VAL2 ]</li><li><b>Pattern 518</b> #PRIORITY-2 XDATE MTH !MTH1 CYR !VAL1 ENDPT &lt;- [!MTH1 ∼, !VAL1 ] <b>Pattern 519</b></li><li>#PRIORITY-2 XDATE RDAY 0 ENDPT &lt;- TODAY <b>Pattern 520</b> #PRIORITY-2 XDATE RDAY -1 ENDPT &lt;- YESTERDAY</li><li><b>Pattern 521</b> #PRIORITY-2 XDATE RWEEK 0 ENDPT &lt;- [ THIS WEEK ]</li><li><b>Pattern 522</b> #PRIORITY-2 XDATE RWEEK -1 ENDPT &lt;- [LAST WEEK ]</li><li><b>Pattern 523</b> #PRIORITY-2 XDATE RMTH 0 ENDPT &lt;- {[THIS MONTH ] MID}</li><li><b>Pattern 524</b> #PRIORITY-2 XDATE RMTH -1 ENDPT &lt;- [ LAST MONTH ] <b>Pattern 525</b> #PRIORITY-2 XDATE RCYR 0 ENDPT &lt;- {[THIS YEAR ] YTD}</li><li><b>Pattern 526</b> #PRIORITY-2 XDATE RQTR 0 ENDPT &lt;- [ THIS QUARTER ]</li><li><b>Pattern 527</b> #PRIORITY-2 XDATE RQTR -1 ENDPT &lt;- [ LAST QUARTER ]</li><li><b>Pattern 528</b> #PRIORITY-2 XDATE RQTR - !VAL1 ENDPT &lt;- [!VAL1 QUARTERS AGO ]</li><li><b>Pattern 529</b> #PRIORITY-2 XDATE RQTR - !VAL1 POINT2 RQTR -1 ENDPT &lt;- [ LAST !VAL1 QUARTERS ]</li><li><b>Pattern 530</b> #PRIORITY-2 XDATE RCYR -1 ENDPT &lt;- [LAST YEAR ]</li><li><b>Pattern 531</b> #PRIORITY-2 XDATE RDAY - !VAL1 ENDPT &lt;- [!VAL1 DAYS AGO ]</li><li><b>Pattern 532</b> #PRIORITY-2 XDATE RDAY - !VAL1 POINT2 RDAY -1 ENDPT &lt;- [ LAST !VAL1 DAYS]</li><li><b>Pattern 533</b> #PRIORITY-2 XDATE RWEEK - !VAL1 ENDPT &lt;- [!VAL1 WEEKS AGO ]</li><li><b>Pattern 534</b> #PRIORITY-2 XDATE RWEEK - !VAL1 POINT2 RWEEK -1 ENDPT &lt;- [ LAST !VAL1 WEEKS]</li><li><b>Pattern 535</b> #PRIORITY-2 XDATE RMTH - !VAL1 ENDPT &lt;- [!VAL1 MONTHS AGO ]</li><li><b>Pattern 536</b> #PRIORITY-2 XDATE RMTH - !VAL1 POINT2 RMTH -1 ENDPT &lt;- [ LAST !VAL1 MONTHS ]</li><li><b>Pattern 537</b> #PRIORITY-2 XDATE RCYR - !VAL1 ENDPT &lt;- [!VAL1 YEARS AGO ]</li><li><b>Pattern 538</b> #PRIORITY-2 XDATE RCYR - !VAL1 POINT2 RCYR -1 ENDPT &lt;- [ LAST !VAL1 YEARS ]</li><li><b>Pattern 539</b> #PRIORITY-2 !ATT1 XDATE !VAL1 -1 &lt;- [!ATT1 &gt;= XDATE !VAL1 !VAL2]</li><li><b>Pattern 540</b> #PRIORITY-2 !ATT1 XDATE !VAL1 -1 &lt;- [ !ATTI {SINCE &gt; } XDATE !VAL1 !VAL2]</li><li><b>Pattern 541</b> #PRIORITY-2 !ATT1 XDATE -1 !VAL1 &lt;- [!ATT1 &lt;= XDATE !VAL1 !VAL2]</li><li><b>Pattern 542</b> #PRIORITY-2 !ATT1 XDATE -1 !VAL1 &lt;- [!ATT1 {BEFORE &lt;} XDATE !VAL1 !VAL2]</li><li><b>Pattern 543</b> #PRIORITY-2 !ATT1 XDATE !VAL1 !VAL2 &lt;- [!ATT1 = XDATE !VAL1 !VAL2]</li><li><b>Pattern 544</b> #PRIORITY-2 !ATT1 XDATE !VAL1 !VAL4 &lt;-    {[!ATT1 BETWEEN XDATE !VAL1 !VAL2 AND XDATE !VAL3 !VAL4 ]    [!ATT1 FROM XDATE !VAL1 !VAL2 TO XDATE !VAL3 !VAL4]}</li><li><b>Pattern 545</b> #PRIORITY-2 SUM &lt;- {TOTAL [SUM OF]}</li><li><b>Pattern 546</b> #PRIORITY-2 COUNT &lt;- [ HOW MANY ]</li><li><b>Pattern 547</b> #PRIORITY-2 AVG &lt;- {AVERAGE AVE}</li><li><b>Pattern 548</b> #PRIORITY-2 MIN &lt;- MINIMUM</li><li><b>Pattern 549</b> #PRIORITY-2 MAX &lt;- MAXIMUM</li><li><b>Pattern 550</b> #PRIORITY-2 !!FUNCTION ( !ATT1) &lt;- [ !!FUNCTION !ATT1]</li><li><b>Pattern 551</b> #PRIORITY-1 SELECT COUNT &lt;- [ COUNT FIRSTWORD ]</li><li><b>Pattern 552</b> #PRIORITY-2 !!FUNCTION1 (!!FUNCTION2 (!ATT1)) &lt;-    {[!!FUNCTION1 !!FUNCTION2 !ATT1]    [!ATT1 !!FUNCTION1 !!FUNCTION2]}</li><li><b>Pattern 553</b> #PRIORITY-0 SELECT COUNT &lt;- [ COUNT FIRSTWORD ]</li><li><b>Pattern 554</b> #PRIORITY-2 COUNT ( DISTINCT !ATT1) &lt;- [ COUNT !ATT1 ]</li><li><b>Pattern 555</b> #PRIORITY-2 COUNT ( !ENT1) &lt;- [ COUNT !ENT1]</li><li><b>Pattern 556</b> !ENT1 WHERE &lt;- [!ENT1 {FOR [ THAT HAVE ] [ THAT HAS ] HAVING}]</li><li><b>Pattern 557</b> #PRIORITY-2 !ATT1 WHERE !ATT1 &lt;- [ WHERE !ATT1 WHERE !ATT1]</li><li><b>Pattern 558</b> #PRIORITY-2 WHERE !ATT1 &lt;- [ WHERE !ATT1 WHERE ]</li><li><b>Pattern 559</b> #PRIORITY-2 !ENT1 WHERE &lt;- [!ENT1 THAT {HAVE HAS}]</li><li><b>Pattern 560</b> #PRIORITY-2 (!ATT1 &gt;= !???1 AND !ATT1 &lt;= !???2) &lt;-    {[!ATT1 BETWEEN !???1 AND !???2]    [!ATT1 FROM !???1 TO j???2]}</li><li><b>Pattern 561</b> #PRIORITY-2 SELECT &lt;- [{WHERE IS DO AM WERE ARE WAS WILL HAD HAS HAVE DID DOES CAN I LIST SHOW GIVE PRINT DISPLAY OUTPUT FORMAT PLEASE RETRIEVE CHOOSE FIND GET LOCATE COMPUTE CALCULATE HOW WHOSE DO WHAT WHO WHEN HOW WHOSE [ WHAT {IS ARE}]} FIRSTWORD ]</li><li><b>Pattern 562</b> NOT NULL &lt;- {[IS NOT NULL] [IS NOT BLANK]}</li><li><b>Pattern 563</b> NULL &lt;- [ IS BLANK ]</li><li><b>Pattern 564</b> = &lt;- IS</li><li><b>Pattern 565</b> #PRIORITY-2 &lt;&gt; &lt;- {NEQ != [ NOT EQUAL ~ TO ]}</li><li><b>Pattern 566</b> &gt;&lt;- {OVER GREATER [ GREATER THAN ] [ MORE THAN ] ABOVE [ NOT LESS THAN OR EQUAL ~ TO ] }</li><li><b>Pattern 567</b> #PRIORITY-2 &gt;=&lt;- {[GREATER THAN OR EQUAL ~ TO] [GT OR EQ ~ TO] [AT LEAST] =&gt; NOT LESS THAN ] [GTE ~ TO ] [MORE THAN OR EQUAL ~ TO]}</li><li><b>Pattern 568</b> #PRIORITY-2 &lt; &lt;- {LESS [ LESS THAN ] BELOW UNDER [ NOT MORE THAN OR EQUAL ~ TO ]}</li><li><b>Pattern 569</b> #PRIORITY-2 &lt;= &lt;- {[LESS THAN OR EQUAL ~ TO ] [ LT OR EQ ~ TO ] [ AT MOST ] =&lt; [NOT MORE THAN] [LTE~ TO]}</li><li><b>Pattern 570</b> #PRIORITY-2 = &lt;- {[EQUAL ~ TO]}</li><li><b>Pattern 571</b> ORDERBY &lt;- [~ AND {BY [ SORTED BY ]}]</li><li><b>Pattern 572</b> #PRIORITY-5 ORDERBY &lt;- [ ORDER BY ]</li><li><b>Pattern 573</b> #PRIORITY-0 DESC &lt;- {DESCENDING {IN {DECREASING DESCENDING} ORDER]}</li><li><b>Pattern 574</b> #PRIORITY-0 ASC &lt;- { ASCENDING [ IN {INCREASING ASCENDING} ORDER ]}</li><li><b>Pattern 575</b> #PRIORITY-0 THATBEGINWITH !??? &lt;- [ BEGINS WITH !???]</li><li><b>Pattern 576</b> #PRIORITY-0 THATENDWITH !??? &lt;- [ ENDS WITH !???]</li><li><b>Pattern 577</b> #PRIORITY-0 THATCONTAIN !??? &lt;- [ CONTAINS !??? ]</li><li><b>Pattern 578</b> #PRIORITY-0 THATSOUNDLIKE !??? &lt;- [ ~ THAT SOUNDS LIKE !??? ]</li><li><b>Pattern 579</b> HAVE &lt;- HAS</li><li><b>Pattern 580</b> #PRIORITY-5 WHERE &lt;- {HAVING [ THAT HAVE ] [ THAT HAS]}</li><li><b>Pattern 581</b> #PRIORITY-5 &gt;= 1 &lt;- [{FOR OF} {ANY SOME}]</li><li><b>Pattern 582</b> #PRIORITY-5 SELECT &lt;- [ SELECT EVERY ]</li><li><b>Pattern 583</b> #PRIORITY-2 !ENT1 WHERE EVERY &lt;- [!ENT1 EVERY ] Pattern 584 EVERY &lt;- {ALL EACH}</li><li><b>Pattern 585</b> !ENT1 WHERE &lt;- [!ENT1 {ARE FOR WITH WHICH HAVE [ THAT HAVE ] [ THAT HAS ]HAVING}]</li><li><b>Pattern 586</b> #PRIORITY-2 !ATT1 EVERY !ENT1 &lt;- [ EVERY !ATT1 ~ OFENTITY!! !ENT1]</li><li><b>Pattern 587</b> &gt;= &lt;- {SINCE FOLLOWING AFTER}</li><li><b>Pattern 588</b> &lt;= &lt;- {BEFORE [PRIOR TO] PRECEEDING}</li><li><b>Pattern 589</b> #PRIORITY-2 WHERE &lt;- [ WHERE WHERE ]</li></ul>

When SQL Generator 20 is initiated, the patterns above are read from an external text file. The patterns are stored in a structure which, for each pattern, contains a priority, the pattern as a text string, and the substitution as a text string. Construction and operation of binding pattern matchers are well known in the art and any of the many techniques that provide the aforementioned capabilities is within the scope of this invention.

2. Conversion of lexical WHERE to internal SQL format

In step 416, a set of patterns is used to convert the Where clause of a query into SQL. These patterns help expand the type of queries the intermediate language can handle, and often map to SQL structures which require subqueries. By adding additional patterns, the intermediate language can be expanded to represent more types of complex SQL queries.

Similarly to the prior pattern matching step, this step compares a text string of the Where clause of a query against a pattern. The Where clause is compared from left to right with the patterns. When there is a match, the matched substring is removed from the Where clause string, and the internal SQL format is supplemented according to the defined substitution. This proceeds until the Where clause string is empty. Patterns are applied in a pre-specified order. Patterns take the form of:    <i>PATTERN[] SUBSTITUTION</i> **

PATTERNS are similar to those in the prior pattern matcher accept that, since there has already been a pass through the prior pattern matcher, there is no need for the {} symbols or the [] phrase symbols. Here, [] and ** are simply symbols used to mark the different portions of the pattern and replacement. In addition to the !???x, !ENTx, !ATTx, !VALx, and !FUNCTIONx binding variables. there are also !!NUM_CONSTRx which are numeric constraint variables that match any numeric constraint (i.e. &gt;, &lt;, ⇐, &gt;=, &lt;&gt;), and NUM_ATTx which match numeric columns.

SUBSTITUTION contains the elements to be added to the internal SQL structure, including SELECT tables, FROM table/alias pairs. WHERE clauses. JOINs. There are also several keywords used in the substitution section. <dl id="dl0021"><dt>BIND !ENTx</dt><dd>The pattern matcher successively matches the Where clause string against the patterns removing portions that have been matched and then matching the remainder. This binds the table held in the binding variable !ENTx to the variable LAST-ENTITY for use in matching the rest of the Where string.</dd><dt>LAST-ENTITY</dt><dd>This contains the last table found in the most recently matched pattern prior to the current pattern match. Thus, this is the last table that is matched in the last pattern that was matched. A pattern can set what the LAST-ENTITY will be for the next matched pattern by using the BIND command. At the start of the pattern matcher, it is set to the last table processed by the SQL generator.</dd><dt>ADD_TO_SELECT !ATTx or !NUM_ATTx</dt><dd>This specifies to add the column in the !ATTx or !NUM_ATTx variable to the SELECT clause of the internal SQL representation.</dd><dt>!ALIASx</dt><dd>In SQL, the FROM clause defines the table from which information comes, and if there is more than one table it generally requires that an alias be assigned to the table for use by the other clauses. The general convention is for an alias to be of the form Tx where x is a number. For example a FROM clause will typically have the format "FROM Customer T1, Order T2" where T1 and T2 are aliases. The SELECT clause may then have the format "SELECT NAME.T1, ORDER_DOLLAR.T2". This prevents confusion if columns from different tables have the same names. When !ALIASx is encountered in the substitution, an alias is generated for storage in the SQL structure. Since there are generally multiple aliases, x is a number. For each different x, a different alias is generated.</dd><dt>PKT !ENTx</dt><dd>This returns either the table in !ENTx or the base table if !ENTx contains a virtual table defined in the conceptual layer.</dd><dt>PK !ENTx</dt><dd>This returns the Primary Key of the table in !ENTx variable.</dd><dt>COL</dt><dd>At this stage, columns stored in !ATTx and !NUM_ATTx variables are fully qualified in the form of table.column. COL !ATTx returns the column portion.</dd><dt>TABLE</dt><dd>TABLE !ATTx or !NUM_ATTx returns the table portion of the fully qualified column name.</dd></dl>

As an example, refer to the following pattern: <ul id="ul0030" list-style="none" compact="compact"><li><i>/* customers with orders of any product ∗/</i></li><li>1 <i>!ENT1 &gt;= 1?ENT2 []</i></li><li><i>2 FROM PKT LAST-ENTITY !ALIAS1</i></li><li><i>3 WHERE EXISTS</i></li><li><i>4   SELECT</i>*</li><li><i>5   FROM PKT !ENT2 !ALIAS2</i></li><li><i>6   WHERE EXISTS</i></li><li><i>7      SELECT</i>*</li><li>8 <i>      FROM PKT !ENT</i>1 <i>!ALIAS3</i></li><li>9 <i>      JOIN !ENT2 !ALIAS2</i></li><li>10 <i>      JOIN LAST-ENTITY !ALIAS1</i></li><li>11 <i>BIND !ENT2</i>**</li></ul>

Line 1 contains the pattern to match -- it will match a string containing "Table_ref1 &gt;= 1 Table ref2" where the table_refs are names of tables or virtual tables stored in the conceptual layer. The string is put in this format during the prior pattern matching and substitution in step 404. Line 2 creates a FROM statement with the base table of the last table referenced by SQL Generator 20 before matching this pattern and create an alias. Line 5 creates a FROM clause with the base table of !ENT2 with an alias distinct from the prior alias. Line 8 is similar to lines 2 and 5. Lines 9 and 10 specify the Joins in internal SQL that will be required. The tables specified in lines 8 and 9 with their respective aliases need to be joined to the table specified in the FROM clause in line 8. Finally, in line 11, !ENT2 is bound as the LAST-ENTITY for any further pattern matching.

Therefore, if the clause being matched contains "ORDERS &gt;= 1 PRODUCTS" and the last table referenced as stored in LAST-ENTITY is CUSTOMERS. the resulting internal SQL format would contain:    FROM CUSTOMERS T1    WHERE EXISTS       SELECT*       FROM PRODUCTS T2       WHERE EXISTS          SELECT*          FROM ORDERS T3          JOIN PRODUCTS T2          JOIN CUSTOMERS T1    BIND PRODUCTS**

The set of patterns employed in step 416 of the illustrated embodiment (i.e., for one instance of an intermediate language) is shown below. <ul id="ul0031" list-style="none"><li><b>Pattern 601</b> /* order date = january 1, 1993 */ !ATT1XDATE !VAL1 !VAL2 [] FROM TABLE !ATT1 !ALIAS1 WHERE !ALIAS1 . COL !ATT1 XDATE !VAL1 !VAL2**</li><li><b>Pattern 602</b> /* balance between 100 and 500 */ !NUM-ATT1 BETWEEN !VAL1 AND !VAL2 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !ALIAS1 . COL !NUM.ATT1 BETWEEN !VAL1 AND !VAL2**</li><li><b>Pattern 603</b> /* balance &gt; 500 */ !NUM-ATT1 !NUM-CONSTR1 !VAL1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !ALIAS1. COL !NUM-ATT1 !NUM-CONSTR1 !VAL1 **</li><li><b>Pattern 604</b> /* customers with orders of every product */ !ENT1 WHERE EVERY !ENT2 [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT *    FROM PKT !ENT2 !ALIAS2    WHERE NOT EXISTS       SELECT*       FROM PKT !ENT1 !ALIAS3       JOIN !ENT2 !ALIAS2    JOIN LAST-ENTITY !ALIAS1 BIND !ENT1 **</li><li><b>Pattern 605</b> /* customers with orders of any product */ !ENT1 &gt;= !ENT2 0 FROM PKT LAST-ENTITY !ALIAS1 WHERE EXISTS    SELECT*    FROM PKT !ENT2 !ALIAS2    WHERE EXISTS          SELECT*          FROM PKT !ENT1 !ALIAS3          JOIN !ENT2 !ALIAS2          JOIN LAST-ENTITY !ALIAS1 BIND !ENT2**</li><li><b>Pattern 606</b> /* salesmen that have at least 1 order */ &gt;= 1 !ENT1 [] FROM PKT LAST-ENTITY !ALIAS1 WHERE EXISTS    SELECT*    FROM PKT !ENT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1             BIND !ENT1 **</li><li><b>Pattern 607</b> /* salesmen that have at least 2 orders */ !NUM-CONSTR1 !VAL1 !ENT1 [] FROM PKT LAST-ENTITY !ALIAS1 WHERE REVERSE !NUM-CONSTR1 !VAL1    SELECT COUNT (*)    FROM PKT !ENTI !ALIAS2    JOIN LAST-ENTITY !ALIAS1             BIND !ENT1 **</li><li><b>Pattern 608</b> /* customers that have every order_date since january 1 */ EVERY !ATT1 XDATE !VAL1 !VAL2 [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM TABLE !ATT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE NOT !ALIAS2 . COL !ATT1 XDATE !VAL1 !VAL2 **</li><li><b>Pattern 609</b> /* salesmen that have every order_amount between 10 and 50 */ EVERY !ATT1 BETWEEN !VAL1 AND !VAL2 [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM TABLE !ATT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE !ALIAS2. COL !ATT1 NOT BETWEEN !VAL1 AND !VAL2 **</li><li><b>Pattern 610</b> * salesmen that have every order_amount &gt; 50 */ EVERY !ATT1 !NUM-CONSTR1 !VAL1 [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM TABLE !ATT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE !ALIAS2. COL !ATT1 REVERSE-PROPER !NUM-CONSTR1 !VAL1 **</li><li><b>Pattern 611</b> /* salesmen that have every cust_name that * sounds like smith */ EVERY !ATT1 THATSOUNDLIKE !??? [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM TABLE !ATT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE SOUNDEX ( !ALIAS2 . COL !ATT1)       &lt;&gt; SOUNDEX ('NOSPACESON !??? ' NOSPACESOFF ) **</li><li><b>Pattern 612</b> /* salesmen that have every cname that contains s */ EVERY !ATT1 THATCONTAIN !??? [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM TABLE !ATT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE NOT !ALIAS2 . COL !ATT1 LIKE ' NOSPACESON %       !??? %' NOSPACESOFF **</li><li><b>Pattern 613</b> /* salesmen that have every cname that begins with s */ EVERY !ATT1 THATBEGINWITH !??? [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM TABLE !ATT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE NOT !ALIAS2 . COL !ATT1 LIKE ' NOSPACESON !??? %'       NOSPACESOFF **</li><li><b>Pattern 614</b> /* salesmen that have every cname that ends with s */ EVERY !ATT1 THATENDWITH !??? [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT *    FROM TABLE !ATTI !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE NOT !ALIAS2 . COL !ATT1 LIKE' NOSPACESON % !???'       NOSPACESOFF **</li><li><b>Pattern 615</b> /* salesmen that have every cname = smith */ EVERY !ATT1 = !??? [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM TABLE !ATT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    WHERE !ALIAS2 . COL !ATT1 &lt;&gt; 'NOSPACESON !??? '       NOSPACESOFF **</li><li><b>Pattern 616</b> /* every order where state = ct */ EVERY !ENT1 WHERE [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT *    FROM PKT !ENT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1 BIND !ENT1 **</li><li><b>Pattern 617</b> /* salary &gt; salary of at least 1 employee */ !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2 &gt;= !ENT1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE ANY       SELECT*       FROM TABLE !NUM-ATT2 !ALIAS2       WHERE !ALIAS1 . COL !NUM.ATT1 &gt;       !ALIAS2 . COL !NUM-ATT2          BIND !ENT1 **</li><li><b>Pattern 618</b> /* salary &gt; salary of at least 6 employees */ !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2 !NUM-CONSTR2 !VAL1 !ENT1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE REVERSE !NUM-CONSTR2 !VAL1       SELECT COUNT (*)       FROM TABLE !NUM-ATT2 !ALIAS2       WHERE !ALIAS1 . COL !NUM-ATT1 !NUM-CONSTR1       !ALIAS2 . COL !NUM-ATT2          BIND !ENT1 **</li><li><b>Pattern 619</b> /* salary &gt; salary of the manager of that employee */ !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2 !ATT1 !ENT1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !NUM-ATT1 !NUM-CONSTR1 ALL       SELECT !NUM-ATT2       FROM TABLE !NUM-ATT2 !ALIAS2       WHERE !ALIAS1 . COL !ATT1 = !ALIAS2 . PK !ENTI          BIND !ENT1 **</li><li><b>Pattern 620</b> /* salary &gt; salary of employees [having name='smith'] */ !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2 WHERE [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !NUM-ATT1 !NUM-CONSTR1 ALL       SELECT !NUM-ATT2       FROM TABLE !NUM-ATT2 !ALIAS2 **</li><li><b>Pattern 621</b> /* salary &gt; salary employees [having name='smith'] */ !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2 !ENT1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !NUM-ATT1 !NUM-CONSTR1 ALL       SELECT !NUM-ATT2       FROM TABLE !NUM-ATT2 !ALIAS2       BIND !ENT1 **</li><li><b>Pattern 622</b> /* salary &gt; salary of every employee [having stale=ct] */ !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2 EVERY !ENT1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !NUM-ATT1 !NUM-CONSTR1 ALL       SELECT !NUM-ATT2       FROM TABLE !NUM-ATT2 !ALIAS2       BIND !ENT1 **</li><li><b>Pattern 623</b> /* salary average salary of employees [having name='smith'] */ !ATT1 !NUM-CONSTR1 !!FUNCTION1 (!ATT2) !ENT1 [] FROM TABLE !ATT1 !ALIAS1 WHERE !ATT1 !NUM-CONSTR1       SELECT !!FUNCTION1 (!ATT2)       FROM TABLE !ATT2 !ALIAS2       BIND !ENT1 **</li><li><b>Pattern 624</b> /* salary &gt; average salary of all employees (having state = ct] */ !ATT1 !NUM-CONSTR1 !!FUNCTION1 ( !ATT2 ) EVERY !ENT1 [] FROM TABLE !ATT1 !ALIAS1 WHERE !ATT1 !NUM-CONSTR1       SELECT !!FUNCTION1 (!ATT2)       FROM TABLE !ATT2 !ALIAS2       BIND !ENT1 **</li><li><b>Pattern 625</b> /* salesman that have the same state */ SAME !ATT1 [] FROM TABLE !ATT1 !ALIAS1 WHERE !ALIAS1 . COL !ATT1 IN       SELECT !ATT1       FROM TABLE !ATT1 !ALIAS2       WHERE !ALIAS1 . PK LAST-ENTITY &lt;&gt;       ALIAS2 . PK LAST-ENTITY **</li><li><b>Pattern 626</b> /* salesmen that have no orders */ NO !ENT1 [] FROM PKT LAST-ENTITY !ALIAS1 WHERE NOT EXISTS    SELECT*    FROM PKT !ENT1 !ALIAS2    JOIN LAST-ENTITY !ALIAS1    BIND !ENT1 **</li><li><b>Pattern 627</b> /* balance &gt; 500 */ !NUM-ATT1 !NUM-CONSTR1 !VAL1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !ALIAS1 . COL !NUM-ATT1 !NUM-CONSTR1 !VAL1 **</li><li><b>Pattern 628</b> /* balance &gt; credit limit */ !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !NUM-ATT1 !NUM-CONSTR1 !NUM-ATT2**</li><li><b>Pattern 629</b> /* balance &gt; (credit limit * 10) */ !NUM-ATT1 !NUM-CONSTR1 !COMP1 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !NUM-ATT1 !NUM-CONSTR1 !COMPI **</li><li><b>Pattern 630</b> /* (balance*5) &gt; (credit limit * 10) */ !COMP1 !NUM-CONSTR1 !COMP2 [] WHERE !COMP1 !NUM-CONSTR1 !COMP2 **</li><li><b>Pattern 631</b> /* balance &gt; sum(order$) */ !NUM-ATT1 !NUM-CONSTR1 !!FUNCTION1 (!NUM.ATT2) [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !NUM-ATT1 !NUM-CONSTR1 !!FUNCTION1 (!NUM-ATT2) **</li><li><b>Pattern 632</b> /* sum(order$) &gt; balance */ !!FUNCTION1 (!NUM.ATT1) !NUM-CONSTR1 !NUM-ATT2 [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !!FUNCTION1 (!NUM-ATT1) !NUM-CONSTR1 !NUM-ATT2**</li><li><b>Pattern 633</b> /* sum(order$) &gt; avg(freight) */ !!FUNCTION1 (!NUM.ATT1) !NUM-CONSTR1 !!FUNCTION2 (!NUM.ATT2) [] FROM TABLE !NUM-ATT1 !ALIAS1 WHERE !!FUNCTION1 (!NUM-ATT1) !NUM-CONSTR1 !!FUNCTION2 (!NUM-ATT2)**</li><li><b>Pattern 634</b> /* customer names that sound like ab */ !ATT1 THATSOUNDLIKE !??? TESTSOUNDEX [] FROM TABLE !ATT1 !ALIAS1 WHERE SOUNDEX (!ATT1)    = SOUNDEX ('NOSPACESON !??? ' NOSPACESOFF ) **</li><li><b>Pattern 635</b> /* customer names that contain ab */ !ATT1 THATCONTAIN !??? [] FROM TABLE !ATT1 !ALIAS1 WHERE !ALIASI . COL !ATT1 LIKE ' NOSPACESON % !??? % 'NOSPACESOFF **</li><li><b>Pattern 636</b> /* customer names that begin with ab */ !ATT1 THATBEGINWITH !??? [] FROM TABLE !ATT1 !ALIAS1 WHERE !ALIAS1 . COL !ATT1 LIKE' NOSPACESON !??? % ' NOSPACESOFF **</li><li><b>Pattern 637</b> /* customer names that end with yz */ !ATT1 THATENDWITH !??? [] FROM TABLE !ATT1 !ALIAS1 WHERE !ALIAS1 . COL !ATT1 LIKE ' NOSPACESON % !??? ' NOSPACESOFF **</li><li><b>Pattern 638</b> /* cust.cnum = ord.cnum */ !ATT1 !NUM-CONSTR1 !ATT2 [] FROM TABLE !ATT1 !ALIAS1 WHERE !ATT1 !NUM-CONSTR1 !ATT2 **</li><li><b>Pattern 639</b> /* state = ct */ !ATT1 = !??? [] FROM TABLE !ATT1 !ALIAS1 WHERE !ALIAS1 . COL !ATT1 = ' NOSPACESON !???' NOSPACESOFF **</li><li><b>Pattern 640</b> /* ( ytd_sales - ytd_cost ) between 100 and 500 */ !COMP1 BETWEEN !VAL1 AND !VAL2 [] WHERE !COMP1 BETWEEN !VAL1 AND !VAL2**</li><li><b>Pattern 641</b> /* (ytd_sales - ytd_cost ) &gt; 500 */ !COMP1 !NUM-CONSTR1 !VAL1 [] WHERE !COMP1 !NUM-CONSTR1 !VALI **</li><li><b>Pattern 642</b> /* customer number = 100 */ !ALPHA-ATT1 !NUM-CONSTR1 !VAL1 [] FROM TABLE !ALPHA-ATT1 !ALIAS1 WHERE !ALIAS1 . COL !ALPHA-ATT1 !NUM-CONSTR1 'NOSPACESON !VAL1 ' NOSPACESOFF</li><li><b>Pattern 643</b> /* customer names &gt; aa */ !ATT1 !NUM-CONSTR1 !??? [] FROM TABLE !ATT1 !ALIAS1 WHERE !ALIAS1 , COL !ATT1 !NUM-CONSTR1 'NOSPACESON !??? ' NOSPACESOFF **</li><li><b>Pattern 644</b> /* names that are null */ !ATT1 NULL [] FROM TABLE ?ATT1 !ALIAS1 WHERE !ALIAS1 . COL !ATT1 IS NULL **</li><li><b>Pattern 645</b> /* names that are not null */ !ATT1 NOT NULL [] FROM TABLE !ATT1 !ALIAS1 WHERE NOT ( !ATT1 IS NULL)**</li><li><b>Pattern 646</b> /* same as product 100 */ = !VAL1 [] WHERE PK LAST-ENTITY = !VAL1 **</li><li><b>Pattern 647</b> /* where salesman have */ !ENT1 WHERE [] BIND !ENT1 **</li><li><b>Pattern 648</b> /* of salesmen */ OF-ENTITY!! !ENT1 WHERE [] BIND !ENT1 **</li><li><b>Pattern 649</b> /* salesman with orders in ct */ !ENT1 [] BIND !ENT1 **</li><li><b>Pattern 650</b> /* and sum(x) between 100 and 500 */ !!FUNCTION1 (!ATT1) BETWEEN !VAL1 AND !VAL2 [] FROM TABLE !ATT1 !ALIAS1 WHERE !!FUNCTION1 (!ALIAS1 . COL !ATT1) BETWEEN !VAL1 AND !VAL2**</li><li><b>Pattern 651</b> /* and count (DISTINCT x) BETWEEN 100 AND 500 */ COUNT ( DISTINCT !ATT1) BETWEEN !VAL1 AND !VAL2 [] FROM TABLE !ATT1 !ALIAS1 WHERE COUNT ( DISTINCT !ATT1) BETWEEN !VAL1 AND !VAL2 **</li><li><b>Pattern 652</b> /* and count (x) BETWEEN 100 AND 500 */ COUNT ( !ATT1) BETWEEN !VAL1 AND !VAL2 [] FROM TABLE !ATT1 !ALIAS1 WHERE COUNT ( !ATT1) BETWEEN !VAL1 AND !VAL2 **</li><li><b>Pattern 653</b> */ and sum(x) &gt; 500 */ !!FUNCTION1 (!ATT1) !NUM-CONSTR1 !VAL1 [] FROM TABLE !ATT1 !ALIAS1 WHERE !!FUNCTION1 (!ALIAS1 . COL !ATT1) !NUM-CONSTR1 !VAL1 **</li><li><b>Pattern 654</b> /* and count (DISTINCT x) &gt; 500 */ COUNT ( DISTINCT !ATT1) !NUM-CONSTR1 !VALI [] FROM TABLE !ATT1 !ALIAS1 WHERE COUNT ( DISTINCT !ALIAS1 . COL !ATT1 ) !NUM-CONSTR1 !VAL1 **</li><li><b>Pattern 655</b> /* and count (x) &gt; 500 */ COUNT ( !ATT1) !NUM-CONSTR1 !VAL1 [] FROM TABLE !ATT1 !ALIAS1 WHERE COUNT ( !ALIAS1 . COL !ATT1 ) !NUM-CONSTR1 !VAL1 **</li><li><b>Pattern 656</b> /* show names of customers with ytd sales */ !NUM-ATT1 [] ADD_TO_SELECT !NUM-ATT1 WHERE !NUM-ATT1 != 0 **</li><li><b>Pattern 657</b> /* show names of customers with states */ !ATT1 [] ADD_TO_SELECT !ATT1 WHERE NOT (!ATT1 IS NULL) **</li></ul>

When SQL Generator <b>20</b> is initiated, the patterns are read from an external text file. The patterns are stored in a structure which, for each pattern contains, the pattern string and the substitution string. Construction and operation of binding pattern matchers are well known in the art and any of the many techniques that provide the aforementioned capabilities is within the scope of this invention. The pattern matcher is recursively called when it encounters nested Where clauses in the case of parentheticals.

C. Join Path

Steps <b>432 - 436</b> call for the computation of join paths, the addition of any new tables to the FROM clause, and inclusion of the explicit joins in the WHERE clause. The computation of the join paths will produce the shortest join path between two tables unless the administrator has defined alternate join paths in the conceptual layer for the user to choose from. With a database structure as shown in <b>Fig. 6</b>, where the direction of the arrows show primary key -&gt; foreign key relationships, the shortest join path can be readily computed as follows.

First, a table of primary key tables is constructed, foreign key tables following all primary key -&gt; foreign key links, the next table in the join chain if not the foreign key table, and the number of joins it takes to get from the primary key table to the foreign key table. This table can be constructed using the foreign key information from the conceptual layer, or by querying the user as to the relationships among the tables.

Second, the navigable paths are computed for the tables to be joined by following the primary key -&gt; foreign key pairs in the table. The Least Detailed Table (LDT) common to the primary key -&gt; foreign key paths of the two tables to join is then found. The LDT is the table up on the graph. In a one-to-many relationship, the one is the least detailed table. If through multiple paths, there are multiple LDTs, the table where the sum of the number of joins is the least is selected. If the number of hops from table to table is equal, it is particularly appropriate for the administrator to define the join paths for user selection. If nothing is defined, one of the paths is arbitrarily chosen. Finally, the join path can be computed by following the primary key -&gt; foreign key relations down to the LDT and then, if necessary, backwards following foreign key -&gt; primary key back up to the second table of the join if neither table is the common LDT.

Using the above procedure, the following table can be constructed for the database of <b>Fig. 6</b>. <tables id="tabl0013" num="0013"><table frame="all"><tgroup cols="4" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="39.37mm" /><colspec colnum="2" colname="col2" colwidth="39.37mm" /><colspec colnum="3" colname="col3" colwidth="39.37mm" /><colspec colnum="4" colname="col4" colwidth="39.37mm" /><thead valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center">Primary table</entry><entry namest="col2" nameend="col2" align="center">Foreign table</entry><entry namest="col3" nameend="col3" align="center">Next Table</entry><entry namest="col4" nameend="col4" align="center">Number of Joins</entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">SALESPEOPLE</entry><entry namest="col2" nameend="col2" align="left">CUSTOMERS</entry><entry namest="col3" nameend="col3" /><entry namest="col4" nameend="col4" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">SALESPEOPLE</entry><entry namest="col2" nameend="col2" align="left">ORDERS</entry><entry namest="col3" nameend="col3" /><entry namest="col4" nameend="col4" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">SALESPEOPLE</entry><entry namest="col2" nameend="col2" align="left">LINE_ITEMS</entry><entry namest="col3" nameend="col3" align="left">ORDERS</entry><entry namest="col4" nameend="col4" align="left">2</entry></row><row><entry namest="col1" nameend="col1" align="left">CUSTOMERS</entry><entry namest="col2" nameend="col2" align="left">ORDERS</entry><entry namest="col3" nameend="col3" /><entry namest="col4" nameend="col4" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">CUSTOMERS</entry><entry namest="col2" nameend="col2" align="left">LINE_ITEMS</entry><entry namest="col3" nameend="col3" align="left">ORDERS</entry><entry namest="col4" nameend="col4" align="left">2</entry></row><row><entry namest="col1" nameend="col1" align="left">ORDERS</entry><entry namest="col2" nameend="col2" align="left">LINE_ITEMS</entry><entry namest="col3" nameend="col3" /><entry namest="col4" nameend="col4" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">VENDORS</entry><entry namest="col2" nameend="col2" align="left">PRODUCTS</entry><entry namest="col3" nameend="col3" /><entry namest="col4" nameend="col4" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">VENDORS</entry><entry namest="col2" nameend="col2" align="left">LINE_ITEMS</entry><entry namest="col3" nameend="col3" align="left">PRODUCTS</entry><entry namest="col4" nameend="col4" align="left">2</entry></row><row><entry namest="col1" nameend="col1" align="left">CODES</entry><entry namest="col2" nameend="col2" align="left">PRODUCTS</entry><entry namest="col3" nameend="col3" /><entry namest="col4" nameend="col4" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">CODES</entry><entry namest="col2" nameend="col2" align="left">LINE_ITEMS</entry><entry namest="col3" nameend="col3" align="left">PRODUCTS</entry><entry namest="col4" nameend="col4" align="left">2</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">PRODUCTS</entry><entry namest="col2" nameend="col2" align="left">LINE_ITEMS</entry><entry namest="col3" nameend="col3" /><entry namest="col4" nameend="col4" align="left">1</entry></row></tbody></tgroup></table></tables>

To find the join path from SALESPEOPLE to ORDERS given the above table, the navigable paths are first computed. For SALESPEOPLE, there are two navigable paths, [SALESPEOPLE CUSTOMER ORDERS LINE_ITEMS] and [SALESPEOPLE ORDERS LINE_ITEMS]. For ORDERS, there is one path [ORDERS LINE_ITEMS]. The common LDT for SALESPEOPLE and ORDERS using either of the paths found for SALESPEOPLE is ORDERS. since there are two paths from SALESPEOPLE to ORDERS, we calculate the number of hops to be one using the [SALESPEOPLE ORDERS] path and two using the [SALESPEOPLE CUSTOMERS ORDERS] path. Without any path definitions in the conceptual layer, the SQL generator will use the shorter path.

As another example, to find the join path from ORDERS to PRODUCTS, the navigable paths are first computed in the same way. This yields the path [ORDERS LINE_ITEMS] for ORDERS, and [PRODUCTS LINE_ITEMS] for PRODUCTS. The common LDT for these paths is LINE_ITEMS. Following the table from ORDERS to LINE_ITEMS and then back up to PRODUCTS, the join path [ORDERS LINE_ITEMS PRODUCTS] is computed. This technique is one of several well know in the art and calculation of the join path is not limited to this technique in the present invention.

In the last example above, the LINE_ITEMS table is introduced in creating the join path. Step <b>436</b> adds any new tables introduced in the process of calculating the join path to the FROM clause in the internal SQL structure. Also included is an alias for the new table.

SQL requires the joins to be explicitly provided in the WHERE clause, and step <b>436</b> implements this. The primary and foreign key columns are stored in the conceptual layer either by the administrator or by Query System <b>1</b> after querying the user. Using the information. the following statement can be included in the WHERE clause to express the join of the above example if the alias for ORDERS, PRODUCTS and LINE_NUMBERS is T1, T2, and T3: "WHERE T1.PRODUCT# = T2.PRODUCTS# AND T2.PRODUCT# = T3.PRODUCT#"

D. Example Conversion of the Intermediate Language to SQL

The example below shows the steps in the conversion of a query in the intermediate language of the form "SHOW CUSTOMER NAME FOR CUSTOMERS THAT HAVE ORDERS OF ANY PRODUCT SORTED BY CUSTOMER CITY" to SQL code.

First, in step <b>402</b>, the query is tokenized into individual units, here marked by &lt;&gt;. <b>&lt;SHOW&gt; &lt;CUSTOMER NAME&gt; &lt;FOR&gt; &lt;CUSTOMERS&gt; &lt;THAT HAVE&gt; &lt;ORDERS&gt; &lt;OF ANY&gt; &lt;PRODUCT&gt; &lt;SORTED BY&gt; &lt;CUSTOMER CITY&gt;</b>

These are the words and phrases made into tokens. This distinction continues, but for purposes of the following steps, the &lt;&gt; around the tokens are not included. Next, in steps <b>404</b> and <b>406</b>, the query is applied against the first set of patterns. The above query matches patterns <b>512, 556, 559, 571</b>, and <b>581</b>.

The patterns are applied in order of priority first and then order of location in the external text file. Since all patterns are either priority 2 or 5, the order in which the patterns above are listed are the order in which they are applied. Therefore, pattern <b>512</b> is applied, and the query becomes: <b>SHOW CUSTOMERS.NAME OFENTITY!! CUSTOMERS WHERE ORDERS OF ANY PRODUCTS SORTED BY CUSTOMERS.CITY</b>

The OFENTITY!! keyword is later used in converting to the internal SQL format and indicates that CUSTOMERS.NAME is a column of entity (table) CUSTOMERS. After the last pattern is applied, patterns <b>556</b> and <b>559</b> no longer match. Also, no new patterns match so patterns <b>571</b> and <b>581</b> remain. By Applying pattern <b>571</b>, which has a higher priority, the query becomes: <b>SHOW CUSTOMERS.NAME OFENTITY!! CUSTOMERS WHERE ORDERS OF ANY PRODUCTS ORDERBY CUSTOMERS.CITY</b>

No new patterns are matched. and there is only one more pattern to match which. when applied, yields:    <b>SHOW CUSTOMERS.NAME OFENTITY!! CUSTOMERS WHERE ORDERS &gt;= 1 PRODUCTS ORDERBY CUSTOMERS. CITY</b>

Steps <b>408 - 412</b> are not applicable to this query, since there is no CREATE VIEW command needed for this type of query. If it were one of a specific set of queries which require CREATE VIEW SQL syntax, SQL Generator <b>20</b> would be called recursively to create the views. Since CREATE VIEW is not necessary, no new words or phrases for conversion were introduced. In step <b>414</b>, the query is broken into SQL components. The query then becomes: <b>SELECT CUSTOMERS.NAME</b><b>WHERE ORDERS &gt;= 1 PRODUCTS</b><b>ORDER BY CUSTOMERS.CITY</b>

The LAST-ENTITY variable is set to CUSTOMERS, since the last table added to the select clause is from the table CUSTOMERS. The OFENTITY!! keyword introduced in the last pattern match is helpful in determining the LAST-ENTITY.

In steps <b>416 - 418</b>, the Where clause "ORDERS &gt;= 1 PRODUCTS" is applied to the patterns shown above, resulting in one match, with pattern <b>605</b>. By applying this pattern, the internal SQL structure for the query becomes: <b>SELECT CUSTOMERS.NAME</b><b>FROM CUSTOMERS T1</b><b>WHERE EXISTS</b>    <b>SELECT*</b>    <b>FROM PRODUCTS T2</b>    <b>WHERE EXISTS</b>       <b>SELECT*</b>       <b>FROM ORDERS T3</b>       <b>JOIN PRODUCTS T2</b>       <b>JOIN CUSTOMERS T1</b><b>ORDER BY CUSTOMERS.CITY</b>

The LAST-ENTITY variable is assigned the table PRODUCTS.

In step <b>420</b>, f there were any table names in the SELECT portion it would expand to include all of the tables columns. Also any virtual table would be expanded. Neither are present in this example, but are performed by simple substitution.

In step <b>422,</b> any columns in the SORT BY portion are added to SELECT if not present. This step converts the SELECT portion of the internal SQL to:    <b>SELECT CUSTOMERS.CITY, CUSTOMERS.NAME</b>

The date conversion function of step <b>424</b> is not applicable, since there are no dates in this example. Similarly, there are no virtual columns for expansion in step <b>426</b>.

If any aliases need to be specified to the FROM clause, they are made in step <b>428.</b> This query created the alias during the application of the Where rules, and no other tables were added to the from clause. The aliases are then substituted into the other sections as well. The internal SQL becomes: <b>SELECT T1.CITY T1.NAME</b><b>FROM CUSTOMERS T1</b><b>WHERE EXISTS</b>    <b>SELECT *</b>    <b>FROM PRODUCTS T2</b>    <b>WHERE EXISTS</b>       <b>SELECT *</b>       <b>FROM ORDERS T3</b>       <b>JOIN PRODUCTS T2</b>       <b>JOIN CUSTOMERS T1</b>    <b>ORDER BY T1.CITY</b>

In step <b>430</b>, the ORDER BY clause is converted to:    <b>ORDER BY 1</b>

In step <b>432,</b> required joins are computed from the internal SQL. They are represented here by the "JOIN Table Alias" statement, and indicates that those tables need to join the table listed in the FROM clause above it. From the prior discussion on join path calculations, the join paths created from the statements:    <b>FROM ORDERS T3</b><b>   JOIN PRODUCTS T2</b><b>   JOIN CUSTOMERS T1</b> are [CUSTOMERS ORDERS] and [ORDERS LINE ITEMS PRODUCTS].

Then, in steps <b>434</b> and <b>436</b>, since the join path calculation introduced a new table, LINE_ITEMS, the table needs to be added to the FROM clause with an alias to make:    <b>FROM ORDERS T3, LINE_ITEMS T4</b>

In step <b>438</b>, the joins are created and added to the where clause from the join paths and foreign key information in the conceptual layer to produce the following Where clause:    <b>WHERE T3.ORDER# = T4.ORDER#</b>       <b>AND T2.PRODUCT# = T4.PRODUCT#</b>       <b>AND T1.CUSTOMER# = T3.CUSTOMER#</b>

Steps <b>440 - 444</b> are not applicable to this example. Finally, in step <b>446</b>, the internal SQL structure, which is now represented as:    <b>SELECT T1.CITY T1.NAME</b>    <b>FROM CUSTOMERS T1</b>    <b>WHERE EXISTS</b>       <b>SELECT *</b>       <b>FROM PRODUCTS T2</b>       <b>WHERE EXISTS</b>          <b>SELECT *</b>          <b>FROM ORDERS T3, LINE_ITEMS T4</b>          <b>WHERE T3.ORDER# = T4.ORDER#</b>           <b>AND T2.PRODUCT# = T4.PRODUCT#</b>             <b>AND Tl.CUSTOMER# = T3.CUSTOMER#</b>    <b>ORDER BY 1</b> is converted to textual SQL. The above representation of the internal SQL structure is in proper textual structure for a query. The process of conversion to the textual query from the internal structure is a trivial step of combining the clauses and running through a simple parser.