TETware Help Pages
Executing a Test Run
Pre-execution criteria
To successfully execute a Test Run there are a number of criteria that MUST be met, they are:
- TET_ROOT must be specified for the local system
Without TET_ROOT specified TETware is unable to find important library files and certain
configuration files.
- TET_SUITE_ROOT should be specified
The rules applicable to TET_SUITE_ROOT are as follows:
- If the path to the Scenario file lies on the same path as TET_ROOT then
TET_SUITE_ROOT does NOT need to be specified.
- If the path to the Scenario file does NOT lie on the same path as TET_ROOT then
TET_SUITE_ROOT MUST be specified.
i.e. if the Scenario file path is /home/testsuite/tests/scen1.scn and TET_ROOT
is /home/tet, then TET_SUITE_ROOT must be specified as /home/testsuite.
- If TET_SUITE_ROOT is specified then the Scenario file must lie on the same path.
i.e. if TET_SUITE_ROOT is /home/testsuite then the path to the Scenario file
must start with /home/testsuite.
- At least one mode of operation MUST be specified
TETware cannot work unless it knows what to do!
- All Environment variables for the Local System MUST be directories
The Environment variables specified for the Local System MUST be directories since
they are general locations where TETware will either look for files or place files.
- Alternate configuration files must exist
If any alternative configuration files are specified by the User then they must
exist.
- If distributed testing is to be performed (i.e. Remote Systems have been added
to the Test Run)
Then the following criteria MUST be met for each Remote System:
- TET_ROOT must be specified for each Remote System.
- TET_TSROOT must be specified for each Remote System.
- One or more scenarios MUST be selected
Testing cannot commence unless TETware knows what it should be testing, so at least one
Scenario must be selected from the Scenario file.
Starting execution
Before execution is due to start the User should check that the criteria outlined above have been met for the Test Run.
Once the criteria above have been met to start execution:
- Use the "Execute Test Run" menu item in the "Run" menu.
- Right-click the "Execution Output" tree node and click the "Execute Test Run" menu item.
Whichever method is chosen to start execution the Confirm execution star dialog
box will be displayed:
If the User chooses "Yes" then TETware will then perform all the necessary checks to ensure that the Test Run can be executed.
If any of the criteria outlined above are NOT met by the configuration for the Test Run then an appropriate dialog box will be displayed indicating the
error (The dialog boxes are not displayed here due to the large number possible).
If all the criteria outlined above ARE met by the configuration for the Test Run then TETware will then generate all the configuration files that are needed by TETware to perform a successful execution. The following files will be generated if TETware
Lite is being used:
- _GUItetbuild.cfg
This file is the build mode file for the Test Run, it is only generated if the User has
selected Build mode as a mode of operation. The location of this file is dependent on
the values of TET_ROOT and TET_SUITE_ROOT, if TET_ROOT is set then the file will be
placed under it, if TET_SUITE_ROOT then it will be placed under there.
- _GUItetexec.cfg
This file is the execute mode file for the Test Run, it is only generated if the User
has selected Execute mode as a mode of operation. The location of this file is dependent
on the values of TET_ROOT and TET_SUITE_ROOT, if TET_ROOT is set then the file will be
placed under it, if TET_SUITE_ROOT then it will be placed under there.
- _GUItetclean.cfg
This file is the clean mode file for the Test Run, it is only generated if the User
has selected Clean mode as a mode of operation. The location of this file is dependent
on the values of TET_ROOT and TET_SUITE_ROOT, if TET_ROOT is set then the file will be
placed under it, if TET_SUITE_ROOT then it will be placed under there.
- _scenTETfile
This is a scenario file that is generated by TETware based upon the scenarios chosen.
The file contains a TETware generated scenario: _TETallscen, this scenario contains the list
of scenarios chosen by the User in the form ^. In this way the chosen
scenarios can be executed by TETware. The location of this file is dependent
on the values of TET_ROOT and TET_SUITE_ROOT, if TET_ROOT is set then the file will be
placed under it, if TET_SUITE_ROOT then it will be placed under there.
Please note: this file is temporary and will be removed after the Test
Run has finished execution, it is mentioned here for completeness.
- tet_code
This file is only generated if the User specifies Custom Result Codes, click
here for more details
on Custom Result Codes. The location of this file is dependent
on the values of TET_ROOT and TET_SUITE_ROOT, if TET_ROOT is set then the file will be
placed under it, if TET_SUITE_ROOT then it will be placed under there.
- systems
This file is only generated if distributed TETware is used and remote systems have been
added to the Test Run. The location of this file is always in the TET_ROOT directory.
- tetdist.cfg
This file is only generated if distributed TETware is used and remote systems have been
added to the Test Run. The location of this file is dependent
on the values of TET_ROOT and TET_SUITE_ROOT, if TET_ROOT is set then the file will be
placed under it, if TET_SUITE_ROOT then it will be placed under there.
It should be noted that for the same Test Run conditions the files are overwritten when the Test Run is re-executed.
Once all the relevant files have been generated the command line to use to execute TETware is built. Once the command line has been built TETware is started with the environment set to the values indicated in the Test Run configuration.
Please note: a 2 second delay has been added to allow time for the Operating System to deliver the captured output
to TETware, it was found during testing that there are times when TETware can finish execution but the Operating System did not deliver the captured output before TETware
had detected that it had finished. The consequence of this was that TETware indicated that it had finished but no output was displayed. To circumvent this a delay was added to enable the output from TETware to be captured correctly.
Notes on the executing Test Run
The "executing" icon will be displayed for the duration that the Test Run is executing.
Whilst the Test Run is executing the menu items relating to Test Run execution will be disabled, that is "Execute Test Run", "Quick Re-run" and "Advanced Re-run". However, the "Stop Test Run" menu item will be enabled.
If TETware starts execution correctly then the "Execution Output" node
icon will change from  to  . This indicates that
TETware is executing.
Whilst executing a Test Run TETware will capture the standard output and standard error streams and display them in the
"Execution Output" and "Error Output" text areas on the right-hand side of TETware respectively.
Once the Test Run has started executing control will be returned to the User so that they can allow the Test Run to execute and continue with the management of other Test Runs etc.
Journal file name
Once the output has been captured TETware will determine what the Journal file name is for this Test Run execution. The Journal file name once captured will be used to display the contents of the Journal file in the Journal text area, please click
here for more information on browsing Journal files. When Test Run execution has finished the Journal file name will be written to the Journal file name in the Report panel so it can then be used for the generation of a Report for the Test Run. For more information on the Report panel click
here. The Journal file name
will also be written to a the Journal file name in the Database Extract Tool,
for more information on writing Journal file parameters to a database please
click here.
Stopping a Test Run
Stopping Test Runs whilst they are executing is NOT recommended since it can undesirable effects on the system. For more information on stopping Test Runs please click
here.
When a Test Run stops?
When a Test Run has finished execution then the "Execution Output" node icon will change from
to . This indicates that the Test Run has finished executing.
Re-executing a Test Run
A Test Run can be re-executed at any time (providing that it is not currently executing). Please note however that the criteria detailed above still hold, i.e. if you change any configuration for the Test Run then it will be used for the re-execution not the previous configuration.
It is also possible to perform partial re-execution of Test Runs, please see the links below for more details:
|
