Introduction
Good test reporting is crucial for serious automation testing. You need an easy way to:
- Check which tests passed, which failed, and why
- Analyze the performance of the tested application
- Compare the results of running the same of different tests
Here we discuss different features of the ATS logging system
Flexible output
Here is where all logs can go
- Test Explorer database
this is a persisted database storage, the results are reviewed by Test Explorer web application. The project is located here - Console
good when reviewing the results of one or a few tests - Mail report
custom
All log outputs are optional. You can use just one of them or some of them as you wish.
Based on log4j
The ATS logging system is currently configured to work along with log4j 1.2. As there are logging "bridges" like slf4j-log4j12 then this is not a big limitation.
Log4j is very flexible in terms of defining how much logging you need by specifying its log level. It is also possible to have more logs from some classes or some packages and less from others.
Note that currently Log4J 2.x is not supported but is planned for future releases.
Configure ATS Database Logging
Here are the steps to make it work for you.
Real-time, but non-blocking the test execution
The logging is performed in real-time. This means while your tests are running, you can look up the results in Test Explorer.
The test execution itself is not blocked waiting for the log event to be persisted in the database. The reason is that each log event is placed into a queue so the test execution can go on. While the test is running an additional dedicated thread moves these cached events to the database.
If you have very heavy logging, it may turn out it will take some time (some number of seconds) until the log event makes it to the database. So we have to say that real-time does not mean exactly immediately
Every participating party can log into the same testcase
Each Java application located anywhere on the test environment can be configured to send its log4j events into the same test case. This means for example that while doing some work with your Application Under Test(AUT), you can have log event from the Java API of the AUT into the same testcase.
Here is who can log:
- The QA from the test
- ATS libraries used in your tests
- Actions running on some ATS agent
- The Application Under Test
- Third-party libraries
Test Explorer user guide
Test Explorer is a web application used to review the test results. It is a separate project from the ATS framework.
Here it is.
Set RUN information
Here we show how to set the information about the RUN of your tests in the Test Explorer UI
Logging the JavaDoc
Here we show how the JavaDoc of the test methods can become a description of your tests in Test Explorer
Specify custom log level on ATS agents
Here are the details on how to have Agent log level different than the Test Executor log level
Increasing the speed we send some events to the log DB
Here we show how to tune the performance of the logging system in case you really need it.
Attach a file to the current testcase
Here we show how to place an important file next to your testcase data in Test Explorer
Printing Test Executor classpath on start
If you want to see(just once) in your console all the jars that are present in the Test Executor classpath, you have to set a system environment variable with the name "ats.log.classpath.on.start" and value "true".
For variable name you can use the AtsSystemProperties.LOG__CLASSPATH_ON_START constant.
Using Test Explorer database logging when running tests from non-java applications
Imagine you have your own tests written in a language different than java, but you would like to use Test Explorer for your needs.
Here we show how to achieve it
What is not supported
Currently, we do not support simultaneous execution of more than one test case.
This is not affecting the performance testing as in that case a single test case causes the need load.
Back to parent page
Go to Table of Contents