Home • Corporate • Contacts • TETware White Paper

Products Solutions Information Datasheet Documentation FAQ Knowledgebase

You are here: TETworks > White Papers > Distributed Testing Managing Distributed Testing Introduction There's a great deal talked about different types of testing: Web testing, regression testing, user testing and of course black box testing, white box testing, even gray box testing. But there is another type of testing that receives less coverage, distributed testing. Here I shall attempt to explain what we mean by distributed testing, and how it compares with non-distributed testing. With TETware the terminology is very important. Running tests on several machines at once is called remote testing, and remote testing is non-distributed. Non-distributed Testing So, let's start with non-distributed testing. It is more frequently used. Non-distributed tests are those that run on a single computer system and do not, normally, involve any form of interaction with other computer systems. I say not, normally, because there are some tests that are run from a host machine to test a target device that is running a real-time or embedded operating system. These tests can be configured and run as non-distributed tests. But the testing of real-time and embedded systems is a subject in itself, and will be the subject of another article. To make it even more complicated, we can divide non-distributed tests into two further categories: local and remote testing. Local testing involves running test cases on a local computer system. So I might run a local test on my laptop, wherever I am. I don't need to be on a network to run a local test. By comparison remote testing does require that I have a network connection. It allows me to use this connection to run a test on another computer system. This is very useful as it allows me to run tests on other processors on the network without leaving my desk, and to collect the results back at my desk. I might run remote tests on several machines, concurrently, (let's call this Simultaneous testing) within one test suite, and collect all of the results back at my desk. With simultaneous testing, even when I'm running tests on many machines at once, those tests do not involve any interaction between the different processors or the tests that they are running. Distributed Testing Distributed testing is different. Because a distributed test case consists of two or more parts that interact with each other. Each part being processed on a different system. It is the interaction between the different test case components that sets distributed testing apart. Typically it is this interaction between different computer systems that is under test. For example, testing a client-server application or the mounting of a file system. All of the test cases processed on all of the different processors contribute towards a single, common, result. This is not the same as simultaneous testing. Because even though simultaneous testing involves different test case components being carried out on different processors, and contributing towards a single result, there is no interaction between the test cases or the processors. As noted above it is this interaction that sets distributed testing apart. A further challenge that we have to face with distributed testing is that of platform. For example testing a client server application may involve using a Windows client to access one or more UNIX servers, and controlling the whole process from a Linux desktop. So our environment has to be written at a level capable of working across all of these platforms. Test Scenarios Once we have set up our hardware environment and established our network connections between the different systems we need to describe the way in which we want to carry out our test cases. We can do this in a test scenario. This lists all of the test cases and describes how they are to be processed. The description is provided in the form of a directive. For the serial processing of test cases on a local machine we don't need any directives, just a list of test cases in the order that they are to be processed. But the test scenario adds a powerful capability to describe tests that, for example, should be repeated a number of times or for a period of time. For remote or distributed testing we use remote or distributed directives which define which systems will run which parts of the test case. Other directives allow us, for example, to run a number of tests in parallel. The power of the directives is further enhanced as we can nest them one inside the other. So, for example, we can use parallel and remote directives to do simultaneous testing, with different tests being carried out on different systems at the same time. The distributed directive is used to define distributed tests, and the scenario file that contains the directive is reads by a Controller (see Figure), which allocates the different parts of the tests to separate control services. One for the local system (the control console), and one for each remote system. Note that these are logical systems and that more than one logical system can be resident on the same physical device. A test suite may comprise many (up to 999) remote systems all of which interact and contribute towards a single test result. We feed in one scenario file and get out one results file. Figure 1: Simple Architecture Diagram for Distributed Testing View Test Scenarios White Paper Synchronization Ensuring that all of the tests happen on all of the systems in the correct order is the greatest challenge in distributed testing. To do this we synchronize the test cases either automatically, at system determined points (e.g. the start and end of each test case), or at user determined points. Synchronization is the key to distributed testing and is important enough to merit separate consideration.  (more . . . ) The challenge of distributed testing lies both in synchronization and the administration of the test process: Configuring the remote systems; generating the scenario files; and processing the results to produce meaningful reports. Being able to repeat the tests consistently, and to select tests for repetition by result i.e. regression testing. And doing this repeatedly and across many different platforms, UNIX, Windows and Linux. Conclusion It's precisely to take the pain away from this process that The Open Group developed TETware a test execution management system that allows users to focus on testing their systems and applications rather than the administrative overhead that comes with the development of local, remote and distributed test suites. TETware allows you to get your test suites developed quicker and deliver your product faster. It takes the tedium out of test suite development and lets developers do what they do best, - developing! Distributed TETware