Evaluating 3G and voice over long term evolution voice quality

BACKGROUND

One factor that customers take into account when selecting a cellular operator is the quality of cellular voice calls. Customers may be willing to pay higher monthly premiums or consume fewer voice, text, or data resources in order to experience higher quality cellular voice calls. Thus, a cellular operator may invest money to purchase resources (e.g., cellular towers and equipment placed thereon) that improve voice call quality on its network. Determining which resources to purchase may be difficult due to challenges in measuring voice call quality and challenges in selecting the resources which are most likely to improve voice call quality per dollar spent. As the foregoing illustrates, new approaches for evaluating cellular voice call quality may be desirable.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures depict one or more implementations in accord with the present teachings, by way of example only, not by way of limitation. In the figures, like reference numerals refer to the same or similar elements.

<figref idref="DRAWINGS">FIG. 1</figref> illustrates an exemplary graph relating cellular voice call quality to a key performance indicator.

<figref idref="DRAWINGS">FIG. 2</figref> illustrates an exemplary computing device for evaluating cellular voice call quality;

<figref idref="DRAWINGS">FIG. 3</figref> illustrates an exemplary process for evaluating cellular voice call quality;

<figref idref="DRAWINGS">FIG. 4</figref> illustrates an exemplary process for creating a model of cellular voice call quality;

<figref idref="DRAWINGS">FIG. 5A</figref> illustrates exemplary cluster classifications for cellular voice call quality;

<figref idref="DRAWINGS">FIG. 5B</figref> illustrates an exemplary process for determining cellular voice call quality based on key performance indicator(s);

<figref idref="DRAWINGS">FIG. 6</figref> is a high-level functional block diagram of an example of a system of networks/devices that provide various communications for mobile stations, for which network traffic may be analyzed and forecasted;

<figref idref="DRAWINGS">FIG. 7</figref> is a high-level functional block diagram of an exemplary non-touch type mobile device which may be involved in analyzing and forecasting network traffic; and

<figref idref="DRAWINGS">FIG. 8</figref> is a high-level functional block diagram of an exemplary touch screen type mobile device which may be involved in analyzing and forecasting network traffic.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. However, it should be apparent that the present teachings may be practiced without such details. In other instances, well known methods, procedures, components, and/or circuitry have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present teachings.

The subject technology provides techniques for evaluating cellular voice call quality. Cellular voice call quality may be measured using a Perceptual Objective Listening Quality Assessment (POLQA) score, also known as ITU-T Rec. P.863. Some implementations of the subject technology are directed toward techniques for calculating the POLQA score or another measure of cellular voice call quality.

According to some aspects of the subject technology, a computing device receives multiple training data points. Each data point in the multiple training data points includes a cellular voice call quality value and values for at least a subset of a set of key performance indicators (KPIs) related to a cellular voice call quality. The cellular voice call quality value may be a POLQA score or another measure of cellular voice call quality. The multiple data points may include measured data points from a Long Term Evolution (LTE) cellular network or any other cellular network or data points measured from a mobile device capable of making voice calls. Exemplary KPIs which may be included in the set of KPIs are illustrated in Table 1 and Table 2, with Table 1 including KPIs measured from an audio sample generated in a LTE network and Table 2 including KPIs measured from a mobile device in a LTE network. Linear correlations to the cellular voice call quality, according to one study, are also listed. However, in other studies, different correlations with the cellular voice call quality may be found.

<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="1"><colspec colname="1" colwidth="217pt" align="center" /><thead><row><entry namest="1" nameend="1" rowsep="1">TABLE 1</entry></row></thead><tbody valign="top"><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row><row><entry>KPIs Measured From Audio Sample.</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="1" colwidth="119pt" align="left" /><colspec colname="2" colwidth="98pt" align="left" /><tbody valign="top"><row><entry>KPI</entry><entry>Correlation to Voice Call Quality</entry></row><row><entry namest="1" nameend="2" align="center" rowsep="1" /></row><row><entry>Audio.Rx.Delta.Delay..ms.</entry><entry>Weak Correlation</entry></row><row><entry>Audio.Rx.Interarrival.Packet.Delay</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.Interarrival.Packet.Delta.Delay</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.Interarrival.Packet.Jitter</entry><entry>Weak Correlation</entry></row><row><entry>Audio.Rx.Jitter..ms.</entry><entry>Weak Correlation</entry></row><row><entry>Audio.Rx.Packet.Count</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.Packet.Loss....</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.Packet.Loss.Count</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.Received.Packet..Byte.</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.Throughput..kbps.</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.Tx.Codec.Delay</entry><entry>Strong Correlation</entry></row><row><entry>Audio.Rx.OPERATOR.Jitter.Delay...ms.</entry><entry>Weak Correlation</entry></row><row><entry>Audio.Tx.Packet.Count</entry><entry>Near No Correlation</entry></row><row><entry>Audio.Tx.Packet.Loss....</entry><entry>Near No Correlation</entry></row><row><entry>Audio.Tx.Packet.Loss.Count</entry><entry>Near No Correlation</entry></row><row><entry>Audio.Tx.Throughput..kbps.</entry><entry>Near No Correlation</entry></row><row><entry>Audio.Tx.Transmitted.Packet..Byte.</entry><entry>Near No Correlation</entry></row><row><entry namest="1" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

<tables id="TABLE-US-00002" num="00002"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="1"><colspec colname="1" colwidth="217pt" align="center" /><thead><row><entry namest="1" nameend="1" rowsep="1">TABLE 2</entry></row></thead><tbody valign="top"><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row><row><entry>KPIs Measured From Mobile Device.</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="1" colwidth="119pt" align="left" /><colspec colname="2" colwidth="98pt" align="left" /><tbody valign="top"><row><entry>KPI</entry><entry>Correlation to Voice Call Quality</entry></row><row><entry namest="1" nameend="2" align="center" rowsep="1" /></row><row><entry>Handover.Happening</entry><entry>Strong Correlation</entry></row><row><entry>LTE.KPI.MAC.DL.Throughput.Mbps. </entry><entry>Strong Correlation</entry></row><row><entry>LTE.KPI.MAC.UL.Throughput.Mbps. </entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.PDCP.DL.Throughput.Mbps.</entry><entry>Strong Correlation</entry></row><row><entry>LTE.KPI.PDCP.UL.Throughput.Mbps.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.PDSCH.BLER...</entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.PDSCH.PRB.Number</entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.PDSCH.Throughput.Mbps.</entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPLPUCCH.Power.dBm.</entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.PUCCH.Power.mW.</entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.PUSCH.BLER...</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.PUSCH.Power.dBm.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.PUSCH.Power.mW.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.PUSCH.PRB.Number</entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.PUSCH.Throughput.Mbps. </entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.RLC.DL.Throughput.Mbps.</entry><entry>Strong Correlation</entry></row><row><entry>LTE.KPI.RLC.UL.Throughput.Mbps.</entry><entry>Weak Correlation</entry></row><row><entry>LTE.KPI.Serving.RSRP.dBm.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.Serving.RSRP.mW.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.Serving.RSRQ.dB.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.Serving.RSRQ.W.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.Serving.RSSI.dBm. </entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.Serving.RSSI.mW.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.SINR.dB.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.SINR.W.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.SRS.Power.dBm.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.SRS.Power.mW.</entry><entry>Near No Correlation</entry></row><row><entry>LTE.KPI.Total.Tx.Power.dBm.</entry><entry>Strong Correlation</entry></row><row><entry>LTE.KPI.Total.Tx.Power.mW.</entry><entry>Strong Correlation</entry></row><row><entry namest="1" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

In some cases, the cellular voice call quality value may be calculated by comparing a local recording of a speaker speaking into a local microphone and a remote recording of the speaker speaking over a cellular connection. The local microphone may be proximate (e.g., within 1 meter or within 5 meters) to a speaker, while the remote recording may be made using a cellular phone call to a destination at which the speaker cannot be heard without the aid of telephone technology. The remote recording may be made, for example, 1 kilometer, 100 kilometers, or 1000 kilometers away from the speaker. The recordings may be played to various test subjects, and the test subject may provide an opinion score as to how “good” and “understandable” the remote recording is, relative to the local recording. For example, the test subject may be asked to rank the goodness or understandability of the remote speaker on a scale of 0 through 10, with 0 representing “I cannot understand the remote speaker at all,” and 10 representing “I can understand the speaker as though he/she were standing right next to me.” A Mean Opinion Score (MOS) of the opinion scores by the tests subjects may be calculated, for example, based on a mean of the 0 through 10 rankings described above.

The computing device determines, for each KPI in the set of KPIs and using the multiple training data points, a linear relationship between the KPI and the cellular voice call quality value. The linear relationship may be determined using linear regression techniques for estimating a best-fit line through a set of points. Several software products, for example, MATLAB®, developed by MathWorks Corporation in Natick, Mass., or SAS/STAT®, developed by SAS Institute Inc. in Cary, N.C., are configured to implement linear regression techniques. The multiple training data points are a set of measured data points of KPI(s) and cellular voice call quality values from an existing cellular network, for example, the cellular network of a specified cellular operator in the United States. Each training data point may include cellular voice call quality value and values for at least a subset of a set of KPIs related to a cellular voice call quality. Each linear relationship is associated with a goodness of fit value. The goodness of fit value measures how well a function estimates values that the function is designed to compute. A “good” goodness of fit indicates that the function accurately estimates values that the function is designed to compute, while a “bad” goodness of fit value indicates that the function does not accurately estimate values that the function is designed to compute. The goodness of fit value may be determined using statistical techniques, for example, a regression analysis technique or a chi-squared technique. One or more compulsory KPIs may be selected based on the goodness of fit. The compulsory KPI(s) may have a goodness of fit value that indicates a better fit (e.g., is greater or less) than a threshold. In some cases, a predetermined number (e.g., two or three) of KPIs having the best fit goodness of fit value may be selected to be the compulsory KPIs. Exemplary techniques for selecting the compulsory KPI(s) are described herein in conjunction with <figref idref="DRAWINGS">FIG. 4</figref>.

According to the regression analysis technique, the goodness of fit value (χ²) is calculated according to Equation 1.

<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mrow><mo>∑</mo><mfrac><msup><mrow><mo>(</mo><mrow><mi>O</mi><mo>-</mo><mi>E</mi></mrow><mo>)</mo></mrow><mn>2</mn></msup><msup><mi>σ</mi><mn>2</mn></msup></mfrac></mrow></mrow></mtd><mtd><mrow><mi>Equation</mi><mo>⁢</mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo>⁢</mo><mn>1</mn></mrow></mtd></mtr></mtable></math></maths><img file="US9119086B1_D0001.tif" />

In Equation 1, O represents an observed value (e.g., of cellular voice call quality), E represents an expected value (e.g., value on the calculated linear relationship between the KPI and the cellular voice call quality value) corresponding to the observed value, and σ represents the standard deviation in the difference between observed and expected values. The summation is taken over all observed values O.

According to the chi-squared technique, the goodness of fit value (χ²) is calculated according to Equation 2.

<maths id="MATH-US-00002" num="00002"><math overflow="scroll"><mtable><mtr><mtd><mrow><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><mfrac><msup><mrow><mo>(</mo><mrow><msub><mi>O</mi><mi>i</mi></msub><mo>-</mo><msub><mi>E</mi><mi>i</mi></msub></mrow><mo>)</mo></mrow><mn>2</mn></msup><msub><mi>E</mi><mi>i</mi></msub></mfrac></mrow></mrow></mtd><mtd><mrow><mi>Equation</mi><mo>⁢</mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo>⁢</mo><mn>2</mn></mrow></mtd></mtr></mtable></math></maths><img file="US9119086B1_D0002.tif" />

In Equation 2, there are n observed values, O_irepresents an observed value (e.g., of cellular voice call quality), and E_irepresents an expected value corresponding to the observed value.

The multiple training data points may be separated into two or more clusters based on the compulsory KPI(s) and the cellular voice call quality values. Exemplary techniques for separating the multiple training data points into clusters and analyzing the clusters are described herein in conjunction with <figref idref="DRAWINGS">FIG. 4</figref>.

<figref idref="DRAWINGS">FIG. 1</figref> illustrates an exemplary graph <b>100</b> relating cellular voice call quality to a compulsory KPI (e.g., Handover.Happening). In the graph <b>100</b>, the compulsory KPI is represented on the horizontal (x) axis and the cellular voice call quality is represented on the vertical (y) axis. The multiple training data points are represented with the curve <b>110</b> on the graph <b>100</b>. The “best fit” line relating the cellular voice call quality to the compulsory KPI is line <b>120</b>. However, the curve <b>110</b> is much better estimated by the multiple joint lines <b>130</b> than by the single line <b>120</b>.

The line <b>120</b> on the graph <b>100</b> represents the linear relationship between the compulsory KPI and the cellular voice call quality that was initially determined by the computing device for the purpose of determining whether the graphed KPI should be a compulsory KPI. However, upon inspection of the graph <b>100</b>, it can be determined that the line <b>120</b> is not the best representation of the relationship between the compulsory KPI and the cellular voice call quality. Rather, the relationship between the compulsory KPI and the cellular voice call quality is best captured by the multiple joint lines <b>130</b>. As shown in <figref idref="DRAWINGS">FIG. 1</figref>, a different linear relationship between the compulsory KPI and the cellular voice call quality exists for each line in the multiple joint lines <b>130</b>.

The computing device determines, for each cluster in the two or more clusters (e.g., clusters corresponding to each line in the multiple joint lines <b>130</b>) and using the multiple training data points, a mathematical relationship for calculating the cellular voice call quality within the cluster based on the one or more compulsory KPIs. The mathematical relationship may be determined using regression techniques. The regression may be linear, quadratic, exponential, logarithmic, etc. Several software products, for example, MATLAB®, developed by MathWorks Corporation in Natick, Mass., or SAS/STAT®, developed by SAS Institute Inc. in Cary, N.C., are configured to implement regression techniques. As shown in <figref idref="DRAWINGS">FIG. 1</figref>, the mathematical relationship may be represented by the multiple joint lines <b>130</b>. In some cases, the mathematical relationship may also be based on other KPIs or other values. The computing device may combine the mathematical relationships for the two or more clusters to generate a module for predicting the cellular voice call quality based on the one or more compulsory KPIs.

The computing device may provide an indication that the module has been generated. The indication may be displayed on the screen of the computing device or provided to another computer via an electronic message (e.g., email, text message, or push notification). The generated module may be used to predict new cellular voice call quality values based on input KPIs. By varying the values of the input KPIs, the generated module may be used to determine which KPIs may be changed (e.g., increased or decreased) to cause a greatest increase in cellular voice call quality. A mobile operator may then invest in changing those KPIs, rather than investing in changing other KPIs, which may not be as effective in improving cellular voice call quality. For example, <figref idref="DRAWINGS">FIG. 1</figref>, which is described in greater detail below, illustrates a relationship between cellular voice call quality and the KPI Handover.Happening. As shown in <figref idref="DRAWINGS">FIG. 1</figref>, as Handover.Happening increases, the voice quality also increases, with the greatest increase occurring at lower levels of Handover.Happening, when Handover.Happening is between 0 and 400. Based on this graph, a mobile operator with a Handover.Happening value of 200 may desire to invest resources in increasing Handover.Happening, while a mobile operator with a Handover.Happening value of 1600 may prefer to invest resources elsewhere, for example, in increasing other KPIs.

The notification regarding whether to invest resources in increasing Handover.Happening or another KPI may be provided over a network (e.g., the network shown in <figref idref="DRAWINGS">FIG. 6</figref>) to a computing device (e.g., a laptop computer or a mobile phone) belonging to an employee of the mobile operator. For example, this information may be displayed to the employee on the employee's screen or provided to the employee via email, text message, or mobile phone push notification. The email, text message, or push notification may be sent from a server to a client computing device (e.g., the laptop computer or the mobile phone). The employee may be employed as a network manager, a network engineer, or a manager of a team of network engineers at the mobile operator.

In <figref idref="DRAWINGS">FIG. 1</figref>, the voice quality curve (curve <b>110</b>) has a concave down slope. If a linear regression (curve <b>120</b>) is simply used to fit and forecast voice quality curve, the error, which is represented by the difference between the fitted value of the regression curve <b>120</b> and the true value on the curve <b>110</b>, is relatively large, as shown by the lines with double arrows in <figref idref="DRAWINGS">FIG. 1</figref>. One manner of avoiding this problem is to split the curve <b>110</b> into multiple pieces, and conduct a separate regression for each piece, as shown by curve <b>130</b>. For each piece, an appropriate regression curve may be applied fit the corresponding piece. This results in a piece-wise learning process in which the concave down curve is clustered into a few pieces (e.g., each joint segment in curve <b>130</b>) by voice quality score. However, in some cases, the total number of clusters (e.g., segments in the curve <b>130</b>) may not have been determined yet. If the number of clusters is large, processing time for separating the curve <b>110</b> may consume a large amount of time or processing resources. However, if the number of clusters is small, the derived model may not be accurate, as shown, for example, by curve <b>120</b> corresponding to a single cluster. The number of clusters, K, may be determined based on the two conditions below.

The first condition concerns classification accuracy, to the degree that the classification model will label a given testing sample to a correct cluster. The classification model may be trained based on training samples for which the voice quality score (and, thus, the associated cluster) and KPI value(s) are known. These training samples may be provided to the classification model for classification. The classification model may classify the training samples to a cluster. The accuracy of the classification may be determined, for example, as a percentage of the training samples that were classified to the correct cluster.

The second condition concerns regression accuracy for each cluster. After a given testing sample is classified to the correct cluster, the regression formula specific for the given cluster may be used to predict the voice quality score for this testing sample. The prediction result will be compared with its true value. The difference between true and predicted value may be measured to determine the regression accuracy.

Thus, the best K for the classification model may be derived. In some cases, a loop with K=2, 3, and 4 may be tested. In a case where the cellular voice call quality value varies between 0 and 5, testing more K values may not be needed or desirable. The details of the K=2, 3, and 4 cluster models are illustrated and described in conjunction with <figref idref="DRAWINGS">FIG. 5A</figref>.

<figref idref="DRAWINGS">FIG. 1</figref> illustrates a single compulsory KPI being related to cellular voice call quality. However, in some implementations, the cellular voice call quality may be a function of multiple (e.g., 2, 3, 4, or more than 4) KPIs. A graph of multiple KPIs and the cellular voice call quality may appear more complex than the graph <b>100</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The graph <b>100</b> provides one non-limiting example to show the relationship between a compulsory KPI and cellular voice call quality associated therewith.

<figref idref="DRAWINGS">FIG. 2</figref> illustrates an exemplary computing device <b>200</b> for evaluating cellular voice call quality. The computing device <b>200</b> may be a laptop computer, a desktop computer, a mobile phone, a tablet computer, a digital music player, a personal digital assistant (PDA), etc. The computing device <b>200</b> may be used to evaluate and improve cellular voice call quality in a network, for example, the mobile traffic network described in conjunction with <figref idref="DRAWINGS">FIG. 6</figref> or a cellular network of a specified mobile operator in the United States or another country. The computing device <b>200</b> may be connected to the mobile traffic network or cellular network or may receive information about the mobile traffic network or cellular network without being directly connected to the network, for example, by manual user input of data related to the cellular network or mobile traffic network.

The computing device <b>200</b> includes a processor <b>205</b>, a network interface <b>210</b>, and a memory <b>215</b>. The computing device <b>200</b> is connected to a display unit <b>295</b>, which may display a graphical user interface (GUI) associated with software or hardware program(s) executing on the computing device <b>200</b>. The display unit <b>295</b> may include one or more monitors, touchscreens, or projectors. The processor <b>205</b> executes programmed instructions, which may be stored in the memory <b>215</b>. While a single processor <b>205</b> is illustrated, the computing device <b>200</b> may have one or multiple processors. If there are multiple processors, the multiple processors may together carry out the functionality of the processor <b>205</b> described herein. The multiple processors may be arranged in processing unit(s), for example, a central processing unit (CPU) or a graphics processing unit (GPU). The network interface <b>210</b> connects the computing device <b>200</b> to a network, for example, the mobile traffic network described in conjunction with <figref idref="DRAWINGS">FIG. 6</figref>. The network interface <b>210</b> may include one or more network interface controllers (NICs). The memory <b>215</b> stores data or instructions. As shown, the memory <b>215</b> stores a data gathering module <b>220</b>, training data points <b>225</b>, a data analysis module <b>230</b>, a cellular voice call quality prediction module <b>235</b>, and a GUI module <b>240</b>.

The data gathering module <b>220</b>, when executed by the processor <b>205</b>, causes the processor <b>205</b> to collect multiple training data points <b>225</b> from a network. For example, the computing device <b>200</b> may measure the training data points <b>225</b> itself, via the network interface <b>210</b>, the computing device <b>200</b> may receive a feed of the training data points <b>225</b> from other computer(s) via a network connection or via a direct connection, or the computing device <b>200</b> may prompt a user to manually input the training data points <b>225</b>. Each data point in the training data points <b>225</b> includes a cellular voice call quality value and values for at least a subset of a set of KPIs related to a cellular voice call quality. Example KPIs which may be in the set of KPIs are described in Table 1. After the training data points <b>225</b> are gathered by the data gathering module <b>220</b>, the training data points <b>225</b> are provided to the data analysis module <b>230</b>. The training data points <b>225</b> may include measured data points from a cellular network, for example, a LTE network.

The data analysis module <b>230</b>, when executed by the processor <b>205</b>, causes the processor <b>205</b> to determine, for each KPI in the set of KPIs and using the training data points <b>225</b>, a linear relationship between the KPI and the cellular voice call quality value. Each linear relationship is associated with a goodness of fit value, which may be calculated using Equation 1 or Equation 2 above. The processor <b>205</b> selects, based on the goodness of fit values, one or more compulsory KPIs. The processor <b>205</b> separates the training data points <b>225</b> into two or more clusters (e.g., corresponding to each line in the multiple joint lines <b>130</b> of <figref idref="DRAWINGS">FIG. 1</figref>) based on the compulsory KPI values or the cellular voice call quality values. The processor <b>205</b> determines, for each cluster in the two or more clusters and using the training data points <b>225</b> in the cluster, a mathematical relationship for calculating the cellular voice call quality within the cluster based on the one or more compulsory KPIs. The processor <b>205</b> generates the cellular voice call quality prediction module <b>235</b>. The cellular voice call quality prediction module <b>235</b> may be used for predicting the cellular voice call quality value by combining the determined mathematical relationships.

The GUI module <b>240</b>, when executed by the processor <b>205</b>, causes the processor <b>205</b> to provide an indication that the cellular voice quality prediction module <b>235</b> has been generated. For example, the GUI module <b>240</b>, when executed, may cause the processor <b>205</b> to provide, at the display unit <b>295</b>, a graph of the relationship(s) between the compulsory KPI(s) and the cellular voice call quality, similar to the graph <b>100</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The provided graph may include a graph of the determined mathematical relationship (e.g., multiple joint lines <b>130</b>) and/or a graph of the training data points <b>225</b>.

As shown in <figref idref="DRAWINGS">FIG. 2</figref>, the components <b>220</b>, <b>225</b>, <b>230</b>, <b>235</b>, and <b>240</b> of the memory <b>215</b> reside on a single computing device <b>200</b>. However, these components <b>220</b>, <b>225</b>, <b>230</b>, <b>235</b>, and <b>240</b> may reside on different computing devices, which may together implement examples of the subject technology. The different computing devices may communicate with one another via a network or via a direct wired or wireless connection.

<figref idref="DRAWINGS">FIG. 3</figref> illustrates an exemplary process <b>300</b> for evaluating cellular voice call quality. The cellular voice call quality may correspond to a POLQA score or to any other measure of cellular voice call quality. The process <b>300</b> begins at step <b>310</b>, where a computing device (e.g., computing device <b>200</b>, executing the data gathering module <b>220</b>, the data analysis module <b>230</b>, and the GUI module <b>240</b>) receives multiple training data points (e.g., training data points <b>225</b>). Each data point in the multiple training data points includes a cellular voice call quality value and values for at least a subset of a set of KPIs related to cellular voice call quality.

In step <b>320</b>, the computing device determines, for each KPI in the set of KPIs and the using the multiple training data points, a linear relationship (e.g., line <b>120</b>) between the KPI and the cellular voice call quality value. The linear relationship may be determined using linear regression techniques, as discussed above. Each linear relationship has a goodness of fit value. The goodness of fit value may be calculated using statistical techniques, for example Equation 1 or Equation 2.

In step <b>330</b>, the computing device selects, based on the goodness of fit values, one or more compulsory KPIs. The one or more compulsory KPIs may correspond to KPIs having a goodness of fit value indicating a better fit than (e.g., being below) a first predetermined threshold. In some cases, the computing device may also select, based on the goodness of fit values, one or more optional KPIs different from the one or more compulsory KPIs. The one or more optional KPIs may correspond to KPIs having a goodness of fit value indicating a worse fit than the first predetermined threshold and a better fit than a second predetermined threshold. In one example, the first predetermined threshold is 0.3 and the second predetermined threshold is 0.5. Alternatively, the first predetermined threshold and the second predetermined threshold may be set such that, out of all of the available KPIs, 15% are a better fit than the first threshold and another 25% are a better fit than the second threshold.

In step <b>340</b>, the computing device separates the multiple training data points into two or more clusters (e.g., corresponding to each line in the multiple joint lines <b>130</b>) based on the compulsory KPI values or the cellular voice call quality values. In some cases, the computing device may separate the multiple training data points into different numbers of clusters, where each of the different numbers is greater than or equal to two (e.g., two clusters, three clusters, and four clusters). The computing device may determine, based on the multiple training data points, which of the numbers of clusters (e.g., two clusters, three clusters, or four clusters) results in a most accurate module for predicting the cellular voice call quality. The determination may be made using artificial intelligence techniques for inspecting the data and determining, based on the inspection of the data, the number of clusters. A most accurate model may have a best (e.g., indicating most accuracy) goodness of fit value, as calculated, for example, by Equation 1 or Equation 2.

In step <b>350</b>, the computing device determines, for each cluster in the two or more clusters and using the multiple training data points in the cluster, a mathematical relationship (e.g., multiple joint lines <b>130</b>) for calculating the cellular voice call quality within the cluster based on the one or more compulsory KPIs. If optional KPIs were selected, the mathematical relationship for calculating the cellular voice call quality may be based on the compulsory KPI(s) and at least one optional KPI. Alternatively, the mathematical relationship may not be based on optional KPIs.

In step <b>360</b>, the computing device generates a module (e.g., cellular voice call quality prediction module <b>235</b>) for predicting the cellular voice call quality value by combining the determined mathematical relationships (e.g., multiple joint lines <b>130</b>). For example, the module may store function(s) corresponding to the determined mathematical relationships. The module may store which KPI(s) are the compulsory KPI(s) and which KPI(s), if any, are the optional KPI(s).

In step <b>370</b>, the computing device provides an indication that the module has been generated. For example, the computing device may present, at a display unit of the computing device, a graph corresponding to output of the module. The graph may include axes for the KPI(s) and the cellular voice call quality value. The graph may include representations of the multiple training data points (e.g., training data points <b>225</b>) and representation of functions corresponding to the generated module (e.g., multiple joint lines <b>130</b>). After step <b>370</b>, the process <b>300</b> ends.

In some examples, the generated module may be used to predict a new cellular voice call quality value based on input KPI(s). The generated module may provide, using the display unit of the computing device or another output unit, an indication of one or more of the input KPI(s) to change for improving the predicted cellular voice call quality value in a case where the predicted cellular voice call quality value is insufficient. The generated module may be provided with price(s) for changing the KPI(s), and indicate, based on the price(s), which KPI(s) should be changed to cause a greatest increase in cellular voice call quality per unit of currency (e.g., dollar) spent. The price(s) for changing the KPI(s) may be input to the computing device by a network engineer based on the network engineer's experience working with vendors on increasing the KPI(s) or based on price quotes from the vendors for increasing the KPI(s).

As set forth above, the steps <b>310</b>-<b>370</b> are carried out according to a specified order. However, the steps may be carried out in any order, not necessarily the one specified above. In some examples, one or more of the steps <b>310</b>-<b>370</b> may be skipped.

<figref idref="DRAWINGS">FIG. 4</figref> illustrates an exemplary process <b>400</b> for creating a model of cellular voice call quality. The process <b>400</b> may be carried out using one or more computers with or without user assistance.

Steps S<b>01</b>-S<b>05</b> of the process <b>400</b> may be referred to as “feature selection.” In these steps, appropriate KPIs are selected for the classification model, which is developed in steps S<b>06</b>-S<b>12</b>.

The process <b>400</b> begins at step S<b>01</b>, where a descriptive study is conducted. In the descriptive study, a pair-wise relationship of cellular voice call quality and KPI(s) may be visualized. The KPI(s) showing high impact on the cellular voice call quality may be selected as independent variable candidates.

In step S<b>02</b>, a linear correlation study is conducted. The Spearman Correlation Algorithm is leveraged to test the linear correlation between the cellular voice call quality and each KPI in the KPI(s). From step S<b>01</b>, it may be determined that the pair-wise relationship between the cellular voice call quality and a given KPI might not be linear. Thus, step S<b>02</b> detects a linear correlation, which is a reference result for step S<b>01</b>.

In step S<b>03</b>, the difference and consistence between the descriptive study of step S<b>01</b> and the linear correlation study of step S<b>02</b> are compared.

In step S<b>04</b>, independent variable candidates are prepared for classification. Based on steps S<b>01</b>, S<b>02</b>, and S<b>03</b>, compulsory KPI independent variable(s) and optional KPI independent variable(s) may be selected. In some cases, there may be m compulsory KPIs and n optional KPIs, where n and m are positive integers.

In step S<b>05</b>, to determine which independent variable combination is the best for the classification, all combinations are populated. The combination among the possible combinations, including all of the compulsory KPIs and none, some, or all of the optional KPIs, that is the “best fit” among the available combinations may be selected. The possible combinations are expressed in Equation 3, where C represents the combinations. <br /><i>C</i>_m^m×(<i>C</i>₀ⁿ<i>+C</i>₁ⁿ<i>+ . . . +C</i>_n-1ⁿ<i>+C</i>_nⁿ)=2ⁿ  Equation 3

Which combination out of the 2ⁿcombinations expressed in Equation 3 is best may be determined by classification accuracy, meaning attempting the classification process with the 2ⁿcombinations, and then selecting the one with least classification error. For example, if n=2, and the first combination has a 90% classification accuracy, while the second combination has a 95% classification accuracy, then the second combination is best.

In step S<b>06</b>, three possible classification scenarios are assumed based on a cellular voice call quality distribution chart developed from the independent variable combinations in step S<b>05</b>.

In step S<b>07</b>, the random forest algorithm is leveraged to train three forests: a two cluster random forest, a three cluster random forest, and a four cluster random forest. The three forests are trained with each of the combinations 1 through 2ⁿdescribed in conjunction with Equation 3.

In step S<b>08</b>, the classification accuracy between the three scenarios—the two cluster random forest, the three cluster random forest, and the four cluster random forest—are compared. For each forest, the highest accuracy among the combinations and the average accuracy among the combinations are determined. It may be determined that the k cluster random forest, where k is a number between two and four, is most accurate based on the k cluster random forest having the greatest average accuracy among the two cluster random forest, the three cluster random forest, and the four cluster random forest.

In step S<b>09</b>, all of the sample data is classified to k clusters based on the k cluster random forest selected in step S<b>08</b>.

In step S<b>10</b>, a regression formula is constructed for each cluster among the k clusters. The cellular voice call quality for each cluster is expressed as a function of the independent variables determined in step S<b>05</b>.

In step S<b>11</b>, the accuracy of the regression for each cluster among the k clusters is evaluated.

In step S<b>12</b>, it is determined that the k cluster classification and regression scenario is optimal. After step S<b>12</b>, a cellular voice call quality value may be determined based on the independent variable KPI(s) as described herein in conjunction with <figref idref="DRAWINGS">FIG. 5B</figref>. After step S<b>12</b>, the process <b>400</b> ends.

As set forth above, the steps S<b>01</b>-S<b>12</b> are carried out according to a specified order. However, the steps may be carried out in any order, not necessarily the one specified above. In some examples, one or more of the steps S<b>01</b>-S<b>12</b> may be skipped.

<figref idref="DRAWINGS">FIG. 5A</figref> illustrates exemplary cluster classifications <b>500</b>A for cellular voice call quality. The classifications <b>500</b>A include table <b>510</b>A and table <b>520</b>A. Table <b>510</b>A shows the cellular voice call quality ranges for each cluster in a two, three, and four cluster scenario. In Table <b>510</b>A, the notation [a, c] refers to a range that is inclusive of a and inclusive of c, in other words, a≦x≦c. The notation [a, c) refers to a range that is inclusive of a and exclusive of c, in other words, a≦x&lt;c. According to table <b>510</b>A, if there are two clusters, the first cluster corresponds to voice quality between 0 and 4, and the second cluster corresponds to voice quality between 4 and 4.5. If there are three clusters, the first cluster corresponds to voice quality between 0 and 3, the second cluster corresponds to voice quality between 3 and 4, and the third cluster corresponds to voice quality between 4 and 4.5. If there are four clusters, the first cluster corresponds to voice quality between 0 and 2, the second cluster corresponds to voice quality between 2 and 3, the third cluster corresponds to voice quality between 3 and 4, and the fourth cluster corresponds to voice quality between 4 and 4.5. Table <b>520</b>A includes a text description of the voice call quality for each cluster in the two, three, and four cluster scenarios. According to table <b>520</b>A, if there are two clusters the first cluster corresponds to imperfect voice quality, and the second cluster corresponds to perfect voice quality. If there are three clusters, the first cluster corresponds to poor voice quality, the second cluster corresponds to good voice quality, and the third cluster corresponds to perfect voice quality. If there are four clusters, the first cluster corresponds to unacceptable voice quality, the second cluster corresponds to moderst voice quality, the third cluster corresponds to good voice quality, and the fourth cluster corresponds to perfect voice quality.

<figref idref="DRAWINGS">FIG. 5B</figref> illustrates an exemplary process <b>500</b>B for determining cellular voice call quality based on key performance indicator(s). The process <b>500</b>B may be implemented after the completion of the process <b>400</b> of <figref idref="DRAWINGS">FIG. 4</figref>. As shown in <figref idref="DRAWINGS">FIG. 5B</figref>, the process <b>500</b>B begins at step <b>510</b>B, where a new measurement is received. In step <b>520</b>B a classification module of a computer classifies the new measurement into one of the k clusters determined using the process <b>400</b>. In step <b>530</b>B, a forecasting algorithm associated with the one of the k clusters is used to forecast the cellular voice call quality. In step <b>540</b>B, the cellular voice call quality is obtained using the forecasting algorithm. After step <b>540</b>B, the process <b>500</b>B ends. The classification module and the regression module, which are described in conjunction with <figref idref="DRAWINGS">FIG. 5B</figref>, may be components of the GUI module <b>240</b> of <figref idref="DRAWINGS">FIG. 2</figref>.

<figref idref="DRAWINGS">FIG. 6</figref> illustrates a system <b>10</b> offering a variety of mobile communication services. The example shows simply two mobile stations (MSs) <b>13</b><i>a </i>and <b>13</b><i>b </i>as well as a mobile communication network <b>15</b>. The stations <b>13</b><i>a </i>and <b>13</b><i>b </i>are examples of mobile stations. The network <b>15</b> provides mobile wireless communications services to those stations as well as to other mobile stations (not shown), for example, via a number of base stations (BSs) <b>17</b>. The network <b>15</b> may correspond to the network of which cellular voice call quality is evaluated using the techniques described herein. The present techniques may be implemented in or used to study any of a variety of available mobile networks <b>15</b>, and the drawing shows only a very simplified example of a few relevant elements of the network <b>15</b> for purposes of discussion here.

The wireless mobile communication network <b>15</b> might be implemented as a network conforming to the code division multiple access (CDMA) IS-95 standard, the 3rd Generation Partnership Project 2 (3GPP2) wireless IP network standard or the Evolution Data Optimized (EVDO) standard, the Global System for Mobile (GSM) communication standard, a time division multiple access (TDMA) standard, the Universal Mobile Telecommunications System (UMTS) standard, the LTE standard belonging to 3GPP or other standards used for public mobile wireless communications. The mobile stations <b>13</b> may be capable of voice telephone communications through the network <b>15</b>. Alternatively or additionally, the mobile stations <b>13</b><i>a </i>and <b>13</b><i>b </i>may be capable of data communications through the particular type of network <b>15</b> (and the users thereof typically will have subscribed to data service through the network).

The network <b>15</b> allows users of the mobile stations such as <b>13</b><i>a </i>and <b>13</b><i>b </i>(and other mobile stations not shown) to initiate and receive telephone calls to each other as well as through the public switched telephone network or “PSTN” <b>19</b> and telephone stations <b>21</b> connected to the PSTN. The network <b>15</b> typically offers a variety of data services via the Internet <b>23</b>, such as downloads, web browsing, email, etc. By way of example, the drawing shows a laptop PC type user terminal <b>27</b> as well as a server <b>25</b> connected to the Internet <b>23</b>; and the data services for the mobile stations <b>13</b> via the Internet <b>23</b> may be with devices like those shown at <b>25</b> and <b>27</b> as well as with a variety of other types of devices or systems capable of data communications through various interconnected networks.

Mobile stations <b>13</b> can take the form of portable handsets, smart-phones or personal digital assistants, although they may be implemented in other form factors. Program applications can be configured to execute on many different types of mobile stations <b>13</b>. For example, a mobile station application can be written to execute on a binary runtime environment for mobile (BREW-based) mobile station, a Windows Mobile based mobile station, Android, I-Phone, Java Mobile, or RIM based mobile station such as a BlackBerry or the like. Some of these types of devices can employ a multi-tasking operating system.

The mobile communication network <b>10</b> can be implemented by a number of interconnected networks. Hence, the overall network <b>10</b> may include a number of radio access networks (RANs), as well as regional ground networks interconnecting a number of RANs and a wide area network (WAN) interconnecting the regional ground networks to core network elements. A regional portion of the network <b>10</b>, such as that serving mobile stations <b>13</b>, can include one or more RANs and a regional circuit and/or packet switched network and associated signaling network facilities.

Physical elements of a RAN operated by one of the mobile service providers or carriers, include a number of base stations represented in the example by the base stations (BSs) <b>17</b>. Although not separately shown, such a base station <b>17</b> can include a base transceiver system (BTS), which can communicate via an antennae system at the site of base station and over the airlink with one or more of the mobile stations <b>13</b>, when the mobile stations are within range. Each base station can include a BTS coupled to several antennae mounted on a radio tower within a coverage area often referred to as a “cell.” The BTS is the part of the radio network that sends and receives RF signals to/from the mobile stations <b>13</b> that are served by the base station <b>17</b>.

The radio access networks can also include a traffic network represented generally by the cloud at <b>15</b>, which carries the user communications and data for the mobile stations <b>13</b> between the base stations <b>17</b> and other elements with or through which the mobile stations communicate. The network can also include other elements that support functionality other than device-to-device media transfer services such as messaging service messages and voice communications. Specific elements of the network <b>15</b> for carrying the voice and data traffic and for controlling various aspects of the calls or sessions through the network <b>15</b> are omitted here form simplicity. It will be understood that the various network elements can communicate with each other and other aspects of the mobile communications network <b>10</b> and other networks (e.g., the public switched telephone network (PSTN) and the Internet) either directly or indirectly.

The carrier will also operate a number of systems that provide ancillary functions in support of the communications services and/or application services provided through the network <b>10</b>, and those elements communicate with other nodes or elements of the network <b>10</b> via one or more private IP type packet data networks <b>29</b> (sometimes referred to as an Intranet), i.e., a private networks. Generally, such systems are part of or connected for communication via the private network <b>29</b>. A person skilled in the art, however, would recognize that systems outside of the private network could serve the same functions as well. Examples of such systems, in this case operated by the network service provider as part of the overall network <b>10</b>, which communicate through the intranet type network <b>29</b>, include one or more application servers <b>31</b> and a related authentication server <b>33</b> for the application service of server <b>31</b>.

A mobile station <b>13</b> communicates over the air with a base station <b>17</b> and through the traffic network <b>15</b> for various voice and data communications, e.g. through the Internet <b>23</b> with a server <b>25</b> and/or with application servers <b>31</b>. Services offered by the mobile service carrier may be hosted on a carrier operated application server <b>31</b>, for communication via the networks <b>15</b> and <b>29</b>. Server such as <b>25</b> and <b>31</b> may provide any of a variety of common application or service functions in support of or in addition to an application program running on the mobile station <b>13</b>. For a given service, an application program within the mobile station may be considered as a ‘client’ and the programming at <b>25</b> or <b>31</b> may be considered as the ‘server’ application for the particular service.

To insure that the application service offered by server <b>31</b> is available to only authorized devices/users, the provider of the application service also deploys an authentication server <b>33</b>. The authentication server <b>33</b> could be a separate physical server as shown, or authentication server <b>33</b> could be implemented as another program module running on the same hardware platform as the server application <b>31</b>. Essentially, when the server application (server <b>31</b> in our example) receives a service request from a client application on a mobile station <b>13</b>, the server application provides appropriate information to the authentication server <b>33</b> to allow server application <b>33</b> to authenticate the mobile station <b>13</b> as outlined herein. Upon successful authentication, the server <b>33</b> informs the server application <b>31</b>, which in turn provides access to the service via data communication through the various communication elements (e.g. <b>29</b>, <b>15</b> and <b>17</b>) of the network <b>10</b>.

The subject technology may be implemented in conjunction with touch screen type mobile stations as well as to non-touch type mobile stations. Hence, our simple example shows the mobile station (MS) <b>13</b><i>a </i>as a non-touch type mobile station and shows the mobile station (MS) <b>13</b> as a touch screen type mobile station. Some implementation may involve at least some execution of programming in the mobile stations as well as implementation of user input/output functions and data communications through the network <b>15</b>, from the mobile stations. Those skilled in the art presumably are familiar with the structure, programming and operations of the various types of mobile stations.

As shown by the above discussion, functions relating to evaluating cellular voice call quality may be implemented on computers connected for data communication via the components of a packet data network, as shown in <figref idref="DRAWINGS">FIG. 6</figref>. For example, the computing device <b>200</b> may correspond to mobile station <b>13</b><i>a</i>, mobile station <b>13</b><i>b</i>, server <b>25</b>, user terminal <b>27</b> or application servers <b>31</b>. Although special purpose devices may be used, such devices also may be implemented using one or more hardware platforms intended to represent a general class of data processing device commonly used to run “server” programming so as to implement the functions discussed above, albeit with an appropriate network connection for data communication.

As known in the data processing and communications arts, a general-purpose computer typically comprises a central processor or other processing device, an internal communication bus, various types of memory or storage media (RAM, ROM, EEPROM, cache memory, disk drives etc.) for code and data storage, and one or more network interface cards or ports for communication purposes. The software functionalities involve programming, including executable code as well as associated stored data. In operation, the code is stored within the general-purpose computer platform. At other times, however, the software may be stored at other locations and/or transported for loading into the appropriate general-purpose computer system. Execution of such code by a processor of the computer platform enables the platform to implement the methodology described herein.

<figref idref="DRAWINGS">FIG. 7</figref> provides a block diagram illustration of an exemplary non-touch type mobile device <b>13</b><i>a</i>, which may correspond to the mobile device <b>13</b>. Although the mobile device <b>13</b><i>a </i>may be a smart-phone or may be incorporated into another device, such as a personal digital assistant (PDA) or the like, for discussion purposes, the illustration shows the mobile device <b>13</b><i>a </i>is in the form of a handset. The handset embodiment of the mobile device <b>13</b><i>a </i>functions as a normal digital wireless telephone station. For that function, the mobile device <b>13</b><i>a </i>includes a microphone <b>402</b> for audio signal input and a speaker <b>404</b> for audio signal output. The microphone <b>402</b> and speaker <b>404</b> connect to voice coding and decoding circuitry (vocoder) <b>406</b>. For a voice telephone call, for example, the vocoder <b>406</b> provides two-way conversion between analog audio signals representing speech or other audio and digital samples at a compressed bit rate compatible with the digital protocol of wireless telephone network communications or voice over packet (Internet Protocol) communications.

For digital wireless communications, the mobile device <b>13</b><i>a </i>also includes at least one digital transceiver (XCVR) <b>408</b>. The mobile device <b>13</b><i>a </i>may be configured for digital wireless communications using one or more of the common network technology types. The concepts discussed here encompass embodiments of the mobile device utilizing any digital transceivers that conform to current or future developed digital wireless communication standards. The mobile device may also be capable of analog operation via a legacy network technology.

The transceiver <b>408</b> provides two-way wireless communication of information, such as vocoded speech samples and/or digital information, in accordance with the technology of the network <b>165</b>. The transceiver <b>408</b> also sends and receives a variety of signaling messages in support of the various voice and data services provided via the mobile device <b>13</b><i>a </i>and the communication network. Each transceiver <b>408</b> connects through RF send and receive amplifiers (not separately shown) to an antenna <b>410</b>. The transceiver may also support various types of mobile messaging services, such as short message service (SMS), enhanced messaging service (EMS) and/or multimedia messaging service (MMS).

The mobile device <b>13</b><i>a </i>includes a display <b>418</b> for displaying messages, menus or the like, call related information dialed by the user, calling party numbers, etc. A keypad <b>420</b> enables dialing digits for voice and/or data calls as well as generating selection inputs, for example, as may be keyed-in by the user based on a displayed menu or as a cursor control and selection of a highlighted item on a displayed screen. The display <b>418</b> and keypad <b>420</b> are the physical elements providing a textual or graphical user interface. Various combinations of the keypad <b>420</b>, display <b>418</b>, microphone <b>402</b> and speaker <b>404</b> may be used as the physical input output elements of the graphical user interface (GUI), for multimedia (e.g., audio and/or video) communications. Of course other user interface elements may be used, such as a trackball, as in some types of PDAs or smart phones.

In addition to normal telephone and data communication related input/output (including message input and message display functions), the user interface elements also may be used for display of menus and other information to the user and user input of selections.

A microprocessor <b>412</b> serves as a programmable controller for the mobile device <b>13</b><i>a</i>, in that it controls all operations of the mobile device <b>13</b><i>a </i>in accordance with programming that it executes, for all normal operations, including those under consideration here. In the example, the mobile device <b>13</b><i>a </i>includes flash type program memory <b>414</b>, for storage of various “software” or “firmware” program routines and mobile configuration settings, such as mobile directory number (MDN) and/or mobile identification number (MIN), etc. The flash type program memory <b>414</b> stores programmed instructions, including those described herein. The mobile device <b>13</b><i>a </i>may include a non-volatile random access memory (RAM) <b>416</b> for a working data processing memory. Of course, other storage devices or configurations may be added to or substituted for those in the example. In a present implementation, the flash type program memory <b>414</b> stores firmware such as a boot routine, device driver software, an operating system, call processing software and vocoder control software, and any of a wide variety of other applications, such as client browser software and short message service software. The memories <b>414</b>, <b>416</b> also store various data, such as telephone numbers and server addresses, downloaded data such as multimedia content, and various data input by the user. Programming stored in the flash type program memory <b>414</b>, sometimes referred to as “firmware,” is loaded into and executed by the microprocessor <b>412</b>.

For purposes of such a discussion, <figref idref="DRAWINGS">FIG. 8</figref> provides a block diagram illustration of an exemplary touch screen type mobile device <b>13</b><i>b</i>. Although configured somewhat differently, at least logically, a number of the elements of the exemplary touch screen type mobile device <b>13</b><i>b </i>are similar to the elements of mobile device <b>13</b><i>a</i>, and are identified by like reference numbers in <figref idref="DRAWINGS">FIGS. 5 and 6</figref>. For example, the touch screen type mobile device <b>13</b><i>b </i>includes a microphone <b>402</b>, speaker <b>404</b> and vocoder <b>406</b>, for audio input and output functions, much like in the earlier example. The mobile device <b>13</b><i>b </i>also includes at least one digital transceiver (XCVR) <b>408</b>, for digital wireless communications, although the mobile device <b>13</b><i>b </i>may include an additional digital or analog transceiver. The concepts discussed here encompass embodiments of the mobile device <b>13</b><i>b </i>utilizing any digital transceivers that conform to current or future developed digital wireless communication standards. As in the mobile device <b>13</b><i>a</i>, the transceiver <b>408</b> provides two-way wireless communication of information, such as vocoded speech samples and/or digital information, in accordance with the technology of the network <b>165</b>. The transceiver <b>408</b> also sends and receives a variety of signaling messages in support of the various voice and data services provided via the mobile device <b>13</b><i>b </i>and the network <b>165</b>. Each transceiver <b>408</b> connects through RF send and receive amplifiers (not separately shown) to an antenna <b>410</b>. The transceiver may also support various types of mobile messaging services, such as short message service (SMS), enhanced messaging service (EMS) and/or multimedia messaging service (MMS).

As in the example of the mobile device <b>13</b><i>a</i>, a microprocessor <b>412</b> serves as a programmable controller for the mobile device <b>13</b><i>b</i>, in that it controls all operations of the mobile device <b>13</b><i>b </i>in accordance with programming that it executes, for all normal operations, and for operations described herein. In the example, the mobile device <b>13</b><i>b </i>includes flash type program memory <b>414</b>, for storage of various program routines and mobile configuration settings. The mobile device <b>13</b><i>b </i>may also include a non-volatile random access memory (RAM) <b>416</b> for a working data processing memory. Of course, other storage devices or configurations may be added to or substituted for those in the example. Hence, outlined above, the mobile device <b>13</b><i>b </i>includes a processor, and programming stored in the flash memory <b>414</b> configures the processor so that the mobile device is capable of performing various desired functions, including the functions described herein.

In the example of <figref idref="DRAWINGS">FIG. 8</figref>, the user interface elements included a display and a keypad. The mobile device <b>13</b><i>b </i>may have a limited number of key(s) <b>430</b>, but the user interface functions of the display and keypad are replaced by a touchscreen display arrangement. At a high level, a touchscreen display is a device that displays information to a user and can detect occurrence and location of a touch on the area of the display. The touch may be an actual touch of the display device with a finger, stylus or other object, although at least some touchscreens can also sense when the object is in close proximity to the screen. Use of a touchscreen display as part of the user interface enables a user to interact directly with the information presented on the display.

Hence, the exemplary mobile device <b>13</b><i>b </i>includes a display <b>422</b>, which the microprocessor <b>412</b> controls via a display driver <b>424</b>, to present visible outputs to the device user. The mobile device <b>13</b><i>b </i>also includes a touch/position sensor <b>426</b>. The sensor <b>426</b> is relatively transparent, so that the user may view the information presented on the display <b>422</b>. A sense circuit <b>428</b> sensing signals from elements of the touch/position sensor <b>426</b> and detects occurrence and position of each touch of the screen formed by the display <b>422</b> and sensor <b>426</b>. The sense circuit <b>428</b> provides touch position information to the microprocessor <b>412</b>, which can correlate that information to the information currently displayed via the display <b>422</b>, to determine the nature of user input via the screen.

The display <b>422</b> and touch sensor <b>426</b> (and possibly one or more keys <b>430</b>, if included) are the physical elements providing the textual and graphical user interface for the mobile device <b>13</b><i>b</i>. The microphone <b>402</b> and speaker <b>404</b> may be used as user interface elements for audio input and output.

The structure and operation of the mobile devices <b>13</b><i>a </i>and <b>13</b><i>b</i>, as outlined above, were described to by way of example only.

The subject technology, in some implementations, provides techniques for evaluating cellular voice call quality in a 3G or LTE network. Training data points are received. Each data point includes a call quality value and values of KPIs related to call quality. For each KPI, a linear relationship, having a goodness-of-fit value, between the KPI and the call quality value is determined using the multiple data points. Compulsory KPIs are selected based on the goodness-of-fit values. The training data points are separated into clusters based on the compulsory KPI values and the quality values. For each cluster, a mathematical relationship for calculating the call quality within the cluster is determined based on the one or more compulsory KPIs. A module is generated for predicting the call quality value by combining the determined mathematical relationships. Advantageously, cellular voice call quality may be estimated more accurately and/or at a lower cost than by using other techniques. Also, relationships between KPIs and the cellular voice call quality may be determined, allowing a network engineer of a mobile operator to determine which KPI(s) should be improved in order to cause the greatest improvement in cellular voice call quality.

While the foregoing has described what are considered to be the best mode and/or other examples, it is understood that various modifications may be made therein and that the subject matter disclosed herein may be implemented in various forms and examples, and that the teachings may be applied in numerous applications, only some of which have been described herein. It is intended by the following claims to claim any and all applications, modifications and variations that fall within the true scope of the present teachings.

Unless otherwise stated, all measurements, values, ratings, positions, magnitudes, sizes, and other specifications that are set forth in this specification, including in the claims that follow, are approximate, not exact. They are intended to have a reasonable range that is consistent with the functions to which they relate and with what is customary in the art to which they pertain.

The scope of protection is limited solely by the claims that now follow. That scope is intended and may be interpreted to be as broad as is consistent with the ordinary meaning of the language that is used in the claims when interpreted in light of this specification and the prosecution history that follows and to encompass all structural and functional equivalents. Notwithstanding, none of the claims are intended to embrace subject matter that fails to satisfy the requirement of Sections 101, 102, or 103 of the Patent Act, should may they be interpreted in such a way. Any unintended embracement of such subject matter is hereby disclaimed.

Except as stated immediately above, nothing that has been stated or illustrated is intended or should be interpreted to cause a dedication of any component, step, feature, object, benefit, advantage, or equivalent to the public, regardless of whether it is or is not recited in the claims.

It will be understood that the terms and expressions used herein have the ordinary meaning as is accorded to such teems and expressions with respect to their corresponding respective areas of inquiry and study except where specific meanings have otherwise been set forth herein. Relational terms such as first and second and the like may be used solely to distinguish one entity or action from another without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “a” or “an” does not, without further constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.