In our days there are a lot of testing frameworks and that leads to an obvious confusion when everybody calls the same thing with a different name.

Here we explain the terminology used in ATS. We do not say it is the best one, it is just a working one.

Test hierarchy

Scenario or Test scenario
Represents a Java method annotated with the @Test annotation. The name of the scenario is the name of the java method indeed
Testcase
This is a single execution of a Scenario. Since in most cases we executed a Scenario just once, we can use either one of these terms.

There is a difference in the meaning of Scenario and Testcase when we run data-driven tests. In such case, the java method has some input arguments and the same scenario is executed many times with different input arguments. Then each scenario execution is called a testcase
TODO provide a link to data-driven test
Suite
This is the java class containing all test methods. In other words, a Suite contains all Scenarios defined in its java class

Run
One execution of one or more Suites containing one or more Scenarios

Test environment participants

Test Executor (TE) - the host which manages and runs the tests.
ATS tests can run from Eclipse, Ant, Maven, Shell command, etc.
Application Under Test (AUT) or Tested Application (TA) - the application which functionality we are testing.
It could be a single standalone application or a number of applications working as one.
Usually, the Tested Application is located on a different host than the Test Executor
External Servers - any external application the AUT works with.
For example, this could be some file or mail storage application
ATS Agent - a standalone application used for different tasks on a usually remote host.
Its name does not reveal its purpose and this is because it is used for a variety of tasks. For example:
- Execute some action (code lines) on a remote host. The ATS frameworks does it internally, but the QA can do it as well
- Collect statistical information about some system resources like CPU, Memory, etc.
- Run actions in multiple threads, so the test can achieve good load on the AUT
Test Explorer - a dedicated host which is used to keep, review and analyze the test results
It is consisted of:
- Database storage where all test data is persisted
- The web application used to review and analyze the test results