Method and test tool for verifying the functionality of a software based unit

SWEDEN (12) PATENT (13) C2 (11)

522 408 (19) SE ₍₅₁)

International class ⁷

G06F 11/00

<img file="SE522408C2_D0001.tif" />

PATENT AND REGISTRATION (45) (41) (22) (24) (62) (86) (86) (83)

Patent filed Application widely available The patent application was submitted on expiration date

Application number International filing date

Filing date for European patent application Deposit of microorganism

2004-02-10

2001-10-28

2000-04-27

2000-04-27 (21) Patent Application Number Q001535-4

Application received as:

Swedish patent application completed international patent application with number □ converted European patent application with number (30) Priority information (73) (72) (74) (54) (56) (57)

Assignee

INVENTOR

AGENT

NAME

Microsoft Corp, One Microsoft Way Redmond 98052-6399 WA US Johan Gustavsson, Huddinge SE, Stefan Johansson, Stockholm SE

AWAPATENT AB

Computer program and procedure for automated testing of a computer's functionality

CALLED PUBLICATIONS:

US A 5,809,108

SUMMARY:

The invention relates to a method and a test tool 110 for verifying the functions of a software-based unit 100 provided with an interface 105 for its external communication. According to the invention, prerecorded data is used for reproducing a test case and for verifying a unit of object for the test case. This prerecorded data includes prerecorded input 125 and prerecorded output 126. Pre-recorded input is applied to an interface of the unit and pre-recorded output is compared to the data transmitted from the unit in response to applied pre-recorded input. If the data transmitted from the unit matches the pre-recorded output, the unit's functions have been verified in accordance with a specific test case.

<img file="SE522408C2_D0002.tif" />

The numbers in brackets indicate international identification code, INID code. Letters in clamps indicate international document code.

522 408

SUMMARY OF THE INVENTION The invention relates to a method and a test tool 110 for verifying the functions of a software-based unit 100 which is provided with an interface 105 for its external communication. According to the invention, prerecorded data is used for reproducing a test case and for verifying a unit of object for the test case. This prerecorded data includes prerecorded input 125 and prerecorded output 126.

Pre-recorded input is applied to an interface of the unit and pre-recorded output is compared to the data transmitted from the unit in response to applied pre-recorded input. If the data transmitted from the unit matches the pre-recorded output, the unit's functions have been verified in accordance with a specific test case.

522 408

Technical area

The present invention relates to a method and a test tool for verifying the functionality of a software-based device equipped with an interface for its external communication.

Background of the invention

When developing products, whether they are hardware products or software products or a combination of the two, the products must be thoroughly tested to verify their functionality before the products are released on the intended market.

Often, the verification, or testing, of a product must be performed during all phases of the entire product development cycle. In the fields of telecommunications, computers and software, a product or element which is part of an overall product is often characterized by having one or more interfaces with which it communicates with an environment, which environment is external with respect to the product or in the product element case. defined by the overall product. A substantial part of the verification, or testing, of such a product or element, both of which will hereafter be referred to as a unit, consists of so-called black box testing. The use of the term black box for a device indicates that there is no knowledge of the device's internal mechanisms. Although some of the device's internal mechanisms or behavior were known, in many cases it is more appropriate to view the product as a black box when performing tests and verifying the device's functions. When a unit is tested as a black box, specific input data comes to one or more of

522 408 device interface to result in corresponding output data from the interfaces. One of the most commonly used procedures for testing a software-based device is to consider the device as a black box and to verify its operation during a number of test cases.

The tests that must be performed when verifying the functions of a device when tested as a black box are normally very time consuming. The staff performing the tests usually have to go through a number of different test cases. Each test case involves a high degree of manual interaction, especially during the phase when the output data is manually verified, but also for generating the input data specific to the test case to be performed.

Today, there are several tools for automated input data generation, e.g. test tool MEDUSA, which is a trademark of Microsoft Corporation. However, one major drawback that remains, whether these existing test tools are used or not, is that output data generated by a device when tested as a black box still needs to be manually verified. In addition, when using tools that provide automated input generation, the result is often a large amount of output that must be verified, which is a daunting task for the testing staff.

The above test tools, ie generators of automated input data, have the characteristic that they create new sets of data for each execution. One of the problems associated with verifying output is the need for accurate knowledge of the input data corresponding to said output. If something seems to be wrong in said output, ie If the unit subject to the test does not appear to function as expected, it must be possible to reproduce the subset of the input that revealed the malfunction of the unit. An obvious reason for this is the desire to use such identified and reproduced input data in repeating the same test case on the unit after the unit has been reconstructed or reconfigured. Alternatively it may be

522 408 desirable to perform the same test case on a second unit of the same type as the first to check whether the second unit exhibits the same malfunction or not. Alternatively, it is simply desirable to repeat a test case on a number of different devices of the same type to verify, e.g. during production, they all work according to the design.

Summary of the Invention

The present invention solves at least some of the aforementioned problems associated with testing and verification of a product or part thereof by providing a method and a testing tool with the features defined in the appended claims.

The invention is based on the idea of ​​using pre-recorded data for reproducing a test case and for verifying a unit subject to the test case. Said prerecorded data includes prerecorded input and prerecorded output. Said prerecorded input data is applied to an interface of the unit and said prerecorded output is compared with the data transmitted from the unit in response to said prerecorded input data. If the data transmitted from the unit corresponds to said prerecorded input data, the functionality of the unit in accordance with the specific test case has been verified.

The test case is something that covers a well-defined part of the unit's functionality and intended working methods. The type of device referred to is a device that is implemented with software, or software in combination with hardware, and which provides one or more interfaces for interaction with its environment. In addition, the unit is of the type it is desirable, or necessary, to test as a black box, ie. to test and verify the device through interaction with its interface as if there was no or little knowledge of the device's internal behavior. The reason for this is e.g. to

522 The internal behavior of the 408 unit is at least partially unknown or too complicated or too expensive to describe.

Thus, the present invention and its various aspects relate to the process of verifying the functionality of a software-based device that provides an interface for its external communication.

According to one aspect of the invention, there is provided a method comprising the steps of: a) placing an external test tool in operative communication with the software-based unit; b) applying prerecorded input data relating to a predefined test case on said interface; and c) comparing the output received from said interface with prerecorded output, which prerecorded output has a verified relationship with said prerecorded input.

According to another aspect of the invention, a testing tool is provided which comprises: a) interface means for enabling operational communication with the software-based unit; and b) data processing means for: (i) reading prerecorded input data relating to a predefined test case; (ii) applying said prerecorded input to said software-based device via said interface means; (iii) reading prerecorded output pertaining to said predefined test case, said prerecorded output having a verified relationship to said prerecorded input; and (iv) comparing the output received from said software-based unit via said interface means with said pre-recorded output.

Thus, according to the invention, an external test tool is operatively connected to the software-based unit. The test tool is designed with interface means through which it is possible to communicate with the software-based device and its interface over a data connection established between the two interfaces. The test tool includes data processing means that execute computer instructions to control the operation of the test 522 408 tool during verification and communication with the software-based device. Within the scope of the present invention and the contents of this specification, a software-based device, or sometimes simply referred to as a device, is to be construed as either a software product, a software component comprised of a software and / or hardware product, a hardware product comprising software, or a complete system such as a software product. telecommunications or computer systems that include software. Although the software-based device is described as having an interface, it will be appreciated that the device may comprise multiple interfaces for its communication and thus its interaction with the testing tool of the invention. Input to and output from the software-based device should be interpreted as digitized information transmitted by a digital signal.

Tests of software-based devices can according to the invention be performed in different ways. The recording of input to and output from a device occurs either while the device is communicating with a test tool or while the device is communicating in its normal operating environment, such as with an external device that the test tool will later simulate. In both cases, the actual verification of a unit is performed by the test tool and based on recorded data. The recording can further be performed on an already verified device or it may include a parallel, manual verification of the device. Recorded data is then used either to verify the same device after e.g. a reconstruction or reconfiguration or to verify another unit of the same type as the unit for which data was initially recorded. Testing or development personnel can thus repeat the test cases for which input / output has been recorded over and over again with a limited amount of manual interaction.

If the software-based device that is subject to the authentication communicates with its environment using a packet data protocol, such as TCP / IP or X.25,

522 408 enables the invention to verify not only such things as the sequence number of transmitted packets, but also all other information transmitted by data packets.

Brief description of the drawings

Further features and advantages of the invention will be more readily understood from the following detailed description of exemplary embodiments of the invention when read in conjunction with the accompanying drawings, in which corresponding reference numerals are used for the corresponding features, and in which:

Fig. 1 schematically shows a test tool and the process of verifying a software-based unit in accordance with an embodiment of the invention;

Fig. 2 schematically shows a test tool and the process of verifying a software-based unit in accordance with an alternative embodiment of the invention;

Fig. 3 shows schematically the process of recording input to, and output from, a software-based unit, which input / output is suitable for use in the embodiments referred to in Fig. 1 and Fig. 2;

Fig. 4 with a flow-like diagram shows the process performed when a software-based unit is verified in accordance with one embodiment of the invention;

Fig. 5a shows an exemplary system in which input to and output from a software-based unit is recorded in accordance with another embodiment of the invention;

Fig. 5b shows an exemplary system in which prerecorded input is applied to the software-based unit and where the output from the unit is compared with prerecorded output according to the embodiment previously referred to in Fig. 5a; and Fig. 6 shows an exemplary user interface for a test tool according to an embodiment of the invention.

522 408

Detailed description of the invention

Figure 1 schematically illustrates the process of verifying a software-based unit 100 by means of a test tool 110 in accordance with one embodiment of the present invention. The test tool is connected to the interface (interfaces) 105 of unit 100 whose functionality is to be tested and verified. In Fig. 1, unit 100 is equipped with an interface 105 through which it receives input and transmits output. However, unit 100 may alternatively be equipped with multiple interfaces, e.g. an interface for its input and an interface for its output. The test tool 100 is also connected to a database 120 which contains stored data to be used by the test tool for testing and verification of the unit 100. This data in the database includes stored input 125 and stored output 126.

The test tool 100 of Fig. 1 includes interface means 115 for communicating with unit 100 and its interface 105. It will be appreciated by those skilled in the art that interface means 115 is selected to be compatible with the unit interface 105 for enabling data communication between the test tool 110 and the unit 100 For the same reason, the interface means 115 is controlled by appropriate communication software that is compatible with the communication protocol applied by the interface 100 of the device 100. The communication protocol used by the device interface 105 is any known or proprietary protocol on which the construction of the interface 105 is based. The only requirement for the use of the present invention is that the communication protocol used by interface 105 of unit 100 is well defined and known in advance so as to enable the design of a test tool 110 comprising a compatible interface 115 controlled by compatible communication protocol software.

522 408

The test tool 110 of Fig. 1 further includes data processing means 116 which is programmed to execute computer instructions to effect that test tool 110 performs the operations required by the invention for verification of software-based unit 100. The realization of these operations as computer instructions for data processing means 116 will be operable. for the skilled person. In addition, the test tool 110 provides an interface (not shown), preferably a graphical user interface, to an operator of the test tool to enable interaction with the test tool. This interaction includes, from an operator's view, such things as requesting that the test tool perform certain test cases as well as receiving reports relating to the outcome of the test cases that have been performed.

Upon verification of the software-based device 100, the verification is performed by testing the device in accordance with one or more test cases. Database 120 stores prerecorded input 125 and prerecorded output 126 for these test cases. This stored input data 125 for a specific test case is the data to be input to interface 105 of unit 100 during this specific test. Similarly, stored output 126 for the same test case is the data to be output from interface 105 during the test if unit 100 is functioning properly.

When unit 100 is to be subjected to a specific test case, test tool 110 reads said input 125 corresponding to test case from database 120. It then transmits this input 125 to unit interface 105 via interface means 115 and the communication line (s) interconnecting the two interfaces. Following or parallel to transmitting said input 125, test tool 110 receives output transmitted from unit 100 via interface 105 and said communication line (s) in response to said transmitted input 125. Test tool 110 then reads, or has just read, said output 126 corresponding to the test case. from database 120. Test tool

522 408 compares output received from unit 100 in response to said input 125 for the test case with said output 126 read from database 120 and corresponding to the same test case. If these two sets of outputs are in agreement, unit 100 has passed the test and the unit's functionality associated with the test case has been verified.

With reference to Fig. 2, an alternative embodiment of the present invention is shown schematically.

This embodiment is similar to that described in FIG.

1 and, except as described below with reference to Fig. 2, the elements of Fig. 2 having and corresponding elements of Fig. 1 operate and interact, as previously described with reference to Fig. 1. Therefore, below, only those features and operations that are different from those described in Fig. 1.

In the embodiment referred to in Fig. 2, the test tool 210 differs from the test tool previously described with reference to Fig. 1 in that it also includes a storage means 217, i.e. a computer-readable memory, for storing portions of, and conditions 218 between, said input data. 125 and said output 126.

This storage device will sometimes be referred to as relational memory. The operations for verifying unit 100 correspond to the operations described above with reference to Fig. 1 except as described below.

In the same way as before, test tool 210 reads said input 125 and output 126 from database 120.

Said input 125 is then analyzed and the information necessary to reproduce the original test case is extracted from input 125. It is assumed that said input 125 follows the same well-defined protocol under which interface 100 for unit 100 operates. Thus, a certain set of input to unit 100 will result in a corresponding set of output being transmitted from the unit. However, this set of outputs can

522 408 vary depending on the internal state in which the unit 100 is located when input data is applied to the unit. Thus, if the unit 100 undergoing a test may be in one or more different internal states, it is preferred to ensure that the unit prior to performing the test is in the same internal state as a unit was in when the output of the test case was recorded. . This can be achieved in a number of ways, either by commanding the unit via the test tool 210 and its associated operator interface (not shown), by manual interaction with the unit 100, or by assuring that the test cases are performed in a certain sequence. For this reason, it is preferred to group a number of test cases and execute these as a test case.

In addition, actual output from unit 100 during a test may vary for other reasons. The information received by and / or transmitted from unit 100 may include real-time information, such as time of day or a parameter that specifies the sequence number of a particular piece of information (which is common if unit 100 communicates in accordance with a packet data protocol). Therefore, in the present embodiment, an analysis of input 125 and output 126 read from database 120 is performed. Useful information can thereby be extracted and any unnecessary information removed from this recorded input and / or output.

The extraction / deletion of data is preferably carried out using a software specifically designed for the task and executing within the test tool. Alternatively, this is accomplished by manual interaction using a graphical user interface connected to the test tool. In this case, there will be a manual transfer of relevant input / output to storage means 217 from database 120.

Some of the inputs and outputs that are useful and will be stored in the storage means 217 will have certain relationships to each other. These conditions can be described as a function f, or possibly as several

522 408 functions, which for a given input X give a certain output Y as the result. The function (s) include: constructed in such a way that certain information in the expected output, which the expected output actual output from unit 100 is compared to, is made dependent on some information in said input. Suppose, for example, the unit to be tested communicates over a packet data connection and said input 125 includes the following information fields:

. . . &lt;23:10:45&gt; &lt;17&gt; &lt;character string # 2&gt;. . .

where the first field is a time stamp, the second field a sequence number and the third field some information as payload. The corresponding output 126 has the same field with the following content:

. . . &lt;23:10:55&gt; &lt;19&gt; &lt;character string # 6&gt;. . .

After removing unnecessary information, the output to be stored in the storage means 217 and used for comparison will have the following information in the same field:

. . . &lt;insignificant&gt; &lt;sequence number + 2&gt; &lt;character string # 6&gt;. . .

that is, the sequence number of expected output can be described by a function f which takes the sequence number as input (ie f (sequence number) = sequence number). This also shows the deletion of some information, e.g. real-time information in the form of timestamps, from recorded input / output that is considered irrelevant to an actual test case. In other words, some information is filtered out of the output received from the unit during a test case while the test is being performed.

Thus, for a specific test case, at least a portion of the corresponding input 125 and at least a portion of the output 126 are stored in the storage means 217. Both input data applied

522 408 of the unit as output used for comparison with the unit output in the response is obtained from the storage means 217.

Figure 3 shows schematically the recording of input to and output of a software-based unit in accordance with one embodiment of the invention. This recorded data is suitable for use in the embodiments referred to in Figures 1 and 2.

In Fig. 3, a unit 100 which is identical to or of the same type as the unit to be tested with recorded data is arranged in communication with an authentic external unit 312 with which unit 100 communicates under normal operating conditions. Unit 100 is a unit known to function properly, e.g. as a result of a manual verification in accordance with known techniques. When unit 100 operates in accordance with a predefined test case, communication between the two units is recorded. Data transmitted to unit 100 from external unit 312 is recorded and stored as a set of input data 325 for the test case. Data transmitted from unit 100 is recorded and stored as a set of output 326 for the same test case. The two sets of recorded data are preferably stored in a database (as shown in Figures 1 and 2). The recording of data is carried out in such a way that it does not affect communication between the units.

Alternatively, reference numeral 312 refers to the aforementioned test tool. In this case, the test tool is designed to simulate the aforementioned external device. Input 325 is then simply the test case input that is manually entered via the above user interface, or it is data that has been automatically generated using some known testing tool for such purpose. In addition, output sent from the unit in response to input data is manually verified before it is stored as output 126 in the database.

Whether unit 312 is the aforementioned external unit or test tool, communication between unit 100 and unit 312 will be performed in accordance with a common protocol and made possible by appropriate

522 408 interfaces, 302 and 313, of the communicating parties as previously described.

Fig. 4 shows a diagram resembling a flow chart to show the operations performed when a software-based unit is verified in accordance with one embodiment of the invention. The overall process of verifying a device can be seen as a process that involves three steps. The first two steps involve preparation and the third step the actual verification.

The first stage concerns recording of input and output and has previously been described with reference to Fig. 3. In step 410, recording of input to and output of the unit is started. In step 415, a test case is run either while the software is communicating with its normal operating environment or while the device is communicating with a test tool. During the test case, said input data sent to the unit is stored in an input memory 430 and the output transmitted from the unit into an output memory 425. After the test case has been run, the recording is stopped in step 420.

The second step concerns the analysis of recorded input and output and has been previously described with reference to Fig. 2. This analysis is performed in step 435 and involves extracting useful information and removing unnecessary information from recorded data. Thus, the test tool reads the input memory and the output memory, e.g. from a database, extracts usable data and removes unwanted data, and then stores the remaining input and output in a relational memory 440 within the test tool.

The third step concerns verification of the output transmitted from a unit being tested during a test case and has been previously described with reference to Fig. 2. In step 450, the aforementioned input and output of a test case from relay memory 440 is read by the test tool. Read input data is then sent to the software-based device to be tested. In step 455, the output received from the unit in response to transmitted input data is compared with the output read from the relational memory. About the two sets

522 408 output differs from each other, it is reported in step 470 to an operator of the test tool, or to a log file, that the test case failed. Alternatively, if the two sets of outputs are the same, it is reported in step 460 to the operator / log file that the test case succeeded.

Figures 5a and 5b show an exemplary embodiment of the present invention applied to a specific system. Figure 5a illustrates how input and output from a software-based device is recorded and stored. Fig. 5b exemplifies the steps of applying prerecorded input to the software-based unit and comparing the output of the unit with prerecorded output when the invention is applied to this specific system.

The embodiment referred to in Figures 5a and 5b relates to the verification of a software-based device called Notification Engine (NE) incorporated by the Internet Cellular Smart Access Platform (ISCA, a trademark of Microsoft Corporation). ICSA is a system platform that enables an operator to provide a number of mobile services to its subscribers. E.g. the services of sending and receiving e-mail messages using a mobile station that may also be connected to a portable computer.

A distinctive feature of the ICSA platform is the sending of short messages service (SMS) messages which include notifications related to emails temporarily stored by the system. When an e-mail server comprised by the ICSA platform receives an e-mail message addressed to a specific user, an SMS message with a notification identifying the e-mail message is sent to a mobile station associated with the e-mail address address. The SMS message is sent via an SMS-C (SMS Center) in the cellular communication system. Using the notification information, the user of the mobile station can retrieve the e-mail stored by the system.

522 408

The unit responsible for the ICSA Notification Engine Platform is called the Notification Engine (NE). The NE communicates with the SMS-C over a packet data connection in accordance with a communication protocol used by the SMS-C. It should be noted that different manufacturers of different SMS-C use slightly different protocols for this communication. The differences relate, for example, the use of sequence numbers and timestamps within the data packets. Examples of such different SMS-C protocols are the SMS-C protocols that go under the names SMPP, CIMD, SMS2000, EMI etc. which come from different respective manufacturers. The NE must therefore be adapted to the specific SMS-C with which it is to communicate. One way to achieve this adaptation is to incorporate an internal state machine within the NE which controls parts of the NE's functions so that it becomes compatible with the protocol used by the specific SMS C. The communication between said SMS-C and said NE generally refers to SMS messages. However, some SMS Cs also support an email transmission protocol, such as Simple Mail Transfer Protocol (SMTP).

Advantageously, the NE within the ICSA platform can be verified in accordance with the present invention. The invention allows test personnel to easily repeat certain tests for the same physical NE, or for a number of different NEs of the same construction. In addition, with the present invention, it will be possible to verify new versions / editions of an NE or its included components in a faster and simplified manner with a minimum of manual interaction from the testing staff.

The embodiment described in Figures 5a and 5b can be seen as a specialized example of the embodiment previously described with reference to Figures 2 and 3. Thus, the features and methods not explicitly described below should be read and understood from the descriptions of Figures 2 and 3. .

522 408

In Figure 5a, an ICSA platform 500 is configured to communicate with an external client / server in the form of an SMS-C 510. The parties communicate using a packet data connection in accordance with the TCP / IP protocol. Included in the ICSA platform is a Notification Engine 502 and also a proxy server 504. The proxy server 504 and the NE communicate within the ICSA platform over a TCP / IP connection. The communication between said SMS-C 510 and the NE 502 is thus forwarded by the proxy server 504. For the proxy server to be able to reconnect the client-server connection, it includes both a client part and a server part. Although proxy server 504 is shown as comprised by ICSA platform 500, it is preferable if the proxy server simply executes on the same hardware as the ICSA platform. The proxy server uses a connection to the ICSA platform in the same way as a local host and a regular TCP / IP connection to said SMS-C 510. The reason for including the proxy server is to enable recording of data traffic to / from the NE during a number of defined test cases. While the proxy server forwards data to / from the NE, the said data copies to a buffer whose contents are in turn regularly copied to a file in the database 515. For each test case, input data to NE 502, and corresponding output from NE 502, are thus recorded by the proxy server. 504 and stored in a database 515.

In Fig. 5b, ICSA platform 500 and its including NE 502 are instead configured to communicate with a test tool 550 without interaction from any proxy server. The purpose of this setting is to verify the function of the NE when operating in accordance with a number of predefined test cases. Test tool 550 simulates the operation of the SMS-C in Fig. 5a. The test tool is designed to include a state machine that mimics the behavior of the specific SMS-C that it simulates. Connected to the test tool is also one

522 408 computer 560 which allows an operator to interact with the test tool.

Before an actual test is performed, the test tool 550 reads the corresponding prerecorded input / output from the database 515. The test tool then extracts useful information from this prerecorded input / output and removes all other data, i.e. it parses and filters said input / output, preferably with the help of a software designed for the task. This parsed and filtered input / output is then stored in a memory within test tool 550, or written back to database 515. Alternatively, the parsing and filtering is performed manually by the operator by means of the computer 560 by first reading recorded input / output from database 515 and then writing parsed and filtered data back to database 515.

The following exemplifies the importance of parsing and filtering performed with respect to said input / output. Suppose recorded data (either input or output) has the following content:

certain data [data: 12345] some other data [comment: certain information] further certain data [data: 6789] ...

If useful data is parsed by following the rule that [starts a pars field and] stops a pars field, the parsed data will have the following content:

[data: 12345] [comment: some information] [data: 6789] if parsed data is filtered by following the rule that all data that starts with comments should be deleted, parsed and filtered data, also called analyzed data, will have the following content :

[data: 12345] [data: 6789]

522 408

When the operator initiates a specific test case, the test tool reads analyzed, i.e., parsed and filtered, the input / output of a test case from its internal memory or from the database 515. Analyzed input is transmitted to ICSA platform 500 and the NE 502 to simulate a real traffic case corresponding to test case. When the test tool receives the response from the ICSA platform, ie output from the NE, it compares this data with previously parsed and filtered pre-recorded output. If the two sets of output matches, the features of the NE 502 with respect to the specific test case have been verified, otherwise the NE 502 is malfunctioning.

As stated above, some SMS Cs support an email transmission protocol. If the entire ICSA platform is considered the entity to be verified, the testing tool must be designed in such a way that it can communicate with the ICSA platform and its appropriate interface in accordance with this email transmission protocol. By using the test tool interface, the actual content of an email message can be verified.

Fig. 6 shows an example of a user interface for a test tool according to an embodiment of the invention. The interface is provided e.g. via a computer monitor, such as the computer shown in Fig. 5b, which computer is connected to the test tool. By using this interface, an operator can initiate different test cases and see the results. The operator can also examine pre-recorded input / output as well as the output with which a unit responds to some applied input. The user interface preferably supports two different types of tests, automatic and semi-automatic. In the automatic mode it is possible to start the test by defining which test cases to run and then pressing Run test. The test result is preferably then written to a log file. The automatic mode requires that pre-5222 408 finite tests have been stored in the database and the corresponding analyzed input / output is used. The semi-automatic mode allows a user to manually check the result for each test case. The operator then has the option to accept or reject the test case. This type of testing is suitable for test cases that are not predefined in the database but where the input and / or output of a test case is manually entered by the operator.

Although the invention has been described with reference to specific embodiments thereof, many different alternatives, modifications and the like will be apparent to those skilled in the art. Therefore, the described embodiments are not intended to limit the scope of the invention as defined by the appended claims.

522 408