Home • Corporate • Contacts • TETware White Paper

Products Solutions Information Datasheet Documentation FAQ Knowledgebase

You are here: TETworks > White Papers > Drudge Taking the Drudge out of Test Development Summary The TETware family of tools are Test Execution Management Systems that takes care of the administration, sequencing, reporting and portability of all of the tests that you develop.  Freeing up developers to concentrate on test development and helping testers by providing them with a single, standard, test harness.  Enabling you to deliver your software projects on time and across multiple operating systems. They are all available off-the-virtual-shelf.  Easily accessed by ftp download.  So stop re-inventing the wheel, take the drudge out of test development and use TETware. Administration Whether you consider them to be frameworks, harnesses or execution environments the TETware products all provide a common standard directory structure in which your tests cases are stored, and executed (or run), and results can be collated. But TETware does more than execute tests it enables users to build (or compile) their tests and to reset their environment after testing is completed. So by running the test you don't change anything and can run the test again in the same situation. We call these stages; build, execute, and clean; modes of operation. TETware can be built in two ways, to manage distributed or non-distributed tests.  Tests that are carried out on the users (local) machine, or on one or more remote machine(s) [or target devices] with the results being passed back to the local machine, are termed non-distributed tests. By comparison a distributed test case has parts that execute simultaneously on either the local system and one or more remote systems, or entirely on two or more remote systems. The different parts are synchronized and contribute towards a single result which is collated on the local system. TETware users also benefit from a common GUI for all of their testing.  Moreover the Help facility within the GUI is designed to enable users to add their own pages so that all of the documentation about your tests resides with the documentation in TETware. Sequencing The sequence in which you wish to run your tests may vary with the type of testing that you want to carry out. Verification testing normally involves a single sequence of tests. But for stress testing you may want to carry out the tests in parallel and on different machines. The ability to carry out tests both on a desktop machine and on any other machine to which you have a network connection is a powerful feature. You can run the tests on a remote machine and gather the results back on your local machine all without leaving your desk. Tests may be set up to run repeatedly, for a period of time or for a number of times. Optionally, during each repetition a test may be selected from a list at random. We control the sequence in which you carry out the tests in a scenario file. This is written using a simple but powerful scenario language. Instructions on how tests are to be processed are provided by directives, and directives may be nested inside each other to fully describe a test suite. The scenario file describes what tests are to be processed. How they are to be processed and where they are to be processed. It is a key input to the test controller. At the start of a test run the test controller reads the scenario file. It then executes the test cases described in the scenario file in the manner described in the scenario file, and the results are posted into a results file (see Figure 1). Figure 1: Simple Architecture Diagram Reporting The TETware products use the POSIX result codes determined in IEEE Std 1003.3 1991 (see Table 1). This provides 8 standard result codes. A further 24 are reserved for internal use by TETware, and an additional 96 codes are available for users to use to determine their own results. For example, while the standard result code 1 (FAIL) has an action item "Continue" some users prefer to designate their own result code with an Abort action item to that they do not consume further machine time (not advisable for large test suites with many hundreds or thousands of test purposes). RESULT CODE NUMBER DESCRIPTION ACTION 0 PASS Continue 1 FAIL Continue 2 UNRESOLVED Continue 3 NOTINUSE Continue 4 UNSUPPORTED Continue 5 UNTESTED Continue 6 UNINITIATED Continue 7 NORESULT Continue Table 1: POSIX Standard Result Codes Defined in IEEE Std 1003.3 1991 The results file is called a Journal file. Again is has a pre-determined structure which contains all of the information that is required about the test run: who ran it and when, how it was configured and what the results were. The Journal file is a text file. It is readable, with a little practice, but its highly structured format lends itself to processing by a report writer, or to post the results to a database We can process the Journal File with a Report Writer to generate a more readable report. For example the standard TETware Report Writer will produce reports in either html or text formats. The html reports can include URIs to point to other html pages if required. The Journal file takes a snapshot of what happened when the tests were run, at that time and in that configuration. It does not allow a comparison to be made between different executions of the same test. To permit this TETware GUI Users have the ability to write Journal file parameters to a database. Once these parameters are stored in the database queries can be used to generate comparative reports, or to prepare reports that include data not produced by the TETware system, e.g. to prepare reports that comply to ISO/IEC 9646-5. We can also use our results to help with regression testing, for example we can rerun a TETware GUI Test Run quickly and possibly by using Result Codes (e.g. to rerun those tests that did not pass the first time) or Mode of Operation. Portability While the TETware products were originally developed to provide a common testing environment for UNIX systems the product was deliberately written at a lower level to provide for ease of portability, not just between UNIX Systems, but also between all POSIX.1 compliant operating systems. So that now includes all flavors of Linux and 32-bit Microsoft Windows operating systems. So if you are developing tests for one platform you don't have to take care of the core aspects provided by TETware when you move your application and it's tests to a different platform. Conclusion That's the crux of the matter. The TETware family of tools allow developers to concentrate on test development without having to worry about the administrative aspects of testing. And, testers only need to learn a single, standard test harness. With TETware you can even use the same GUI for all your testing, and the tests can be documented in a common Help System that can record not only how the product operates, but also how your tests need to be run as well.