ATS : Troubleshooting

Introduction

Here you can find explanations of various error messages or issues that may occur.

Reporting issues

Before reporting possible issue please check the section below with possible causes and their resolution. For connection-related issues you test connectivity with external tool like ping and telnet to the remote machine.

If error is still not localized there you may start by adding this info as issue in GitHub issues :

1. Version of the ATS Framework used. Example: 4.0.5;
2. Short scenario of the case; Example: Could not successfully log into TestExplorer database.
3. Detailed info how to reproduce or better an isolated code snippet. Archived simple Maven project would be one of the best options. Example: Paste simple test with any setup/prerequisites if needed like log4j.xml.
4. Other environment details:
   1. OS used and its version;
   2. JavaSE version used for test executor and optionally for involved ATS agents;
   3. DB name (and version) used for TestExplorer DB logging. Version of the DB scripts; Fresh install or upgraded version;
   4. Other involved products like HTTP/FTP servers;

List with known possible errors and troubleshooting instructions

Exception when invoking action on remote agent, ClassNotFound or NoSuchMethod errors in ATS packages

Message sample: Exception during execution of action 'Get Product Version' for component 'my_agent_actions_component' at myhost.mydomain:8089: NoClassDefFoundError com/axway/ats/action/system/SystemOperations
Reason: If ClassNotFound or NoSuchMethodError exception contains errors from packages starting with com/axway/ats then ATS setup seems incomplete (missing required libraries) or there are differences between library( JAR) versions on test executor or ATS agents.
Solution: Upgrade all ATS dependencies to the same version and add any missing ones. For ATS artifact names you may refer to the table in Retrieving ATS libraries

• For Maven: Check pom.xml dependencies of test project. Use either Dependency Hierarchy tab in Eclipse or in console via: mvn dependency:tree > myList.txt to check for expected dependencies.
• Make sure that any agents needed also have the same version installed as the one on Test Executor.
• If exception comes from some ATS agent then check that all additional libraries are in ats-agent/actions_dependencies directory. If some of them are from ATS then their version should match the agent version itself. One such library needed is Action library when process execution or file transfers are needed. If agent custom packages are produced with Maven assembly plugins then you should check and add such dependencies in corresponding assembly.xml file.

Some actions component is not registered

Message: com.axway.ats.agent.core.exceptions.NoSuchComponentException: Component 'ABC' is not registered
Reason: The actions module by the name of 'ABC' is not deployed on the ATS agent;
Solution: There are many reasons for a module not to load, but the most common are:

• see whether you are trying to run tests on a different machine that does not have such a component deployed;
• check the ats-agent/actions directory for the component's jar archive;
• check the agent log (./logs/ subdir) or Web server container's log for any deployment errors;

Internal error running an Agent action

Message: com.axway.ats.agent.core.exceptions.InternalComponentException
Reason: The Agent's action has been halted due to an exception thrown by the tested component;
Solution: Most probably this is an issue with the custom actions' jar/component itself - in such a case it is best to check the issue with the development team, responsible for the component under test;

Queue full

Message: java.lang.IllegalStateException: Queue full
Reason: There are too many requests for logging into TestExplorer logging database. This reason is in case that Java stack trace includes classes from com.axway.ats.log package. It could be either because logging is too verbose or that connection to the database is very slow like DB server resides in another geographical location.
Solution:

• if you are using DEBUG level in log4j.properties/.xml file decrease severity to INFO or WARN for the root logger. On demand, you may increase logging severity just of specific Java package/class. For reference you may check examples here
• use TestExplorer DB in your local network if you think logging severity could not be decreased anymore;
• Check this page for more details

Cannot connect to an ATS Agent instance

Message: com.axway.ats.agent.core.exceptions.AgentException: HTTP transport error: java.net.ConnectException: Connection refused: connect. Check server logs for more details.
Reason: There is no connection to the Agent.
Solution: Adjust correct host and port of the agent to be contacted or check agent logs whether it starts successfully. Check for firewall rules too. You may also get details in the "Verify it runs properly" section in page: The ATS Agent

Unable to encrypt|decrypt message

Message: java.security.InvalidKeyException: Illegal key size
Reason: You do not have set full encryption support, i.e. Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files.
Note that in most recent JRE/JDKs  (like JavaSE 8u162 and newer) those policy files are automatically enabled.
Solution: Download package from http://www.oracle.com/technetwork/java/javase/downloads/ and replace the two jars in

[JRE]/lib/security

Unsupported major.minor version nn.n

Sample message: java.lang.UnsupportedClassVersionError: com/some/package/SomeClassName : Unsupported major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(Unknown Source)
...

Version number may vary like 51.0 and 52.0. Version 51.0 means Java SE 7 byte-code and 52.0 - Java SE 8. Other versions: 53 - Java 9, 55 - Java 11. For reference check link here.
Reason: You are running class file /JAR compiled to be used only with newer JREs. For example JAR compiled with Java 8 JRE and tried to be run with Java 7 JRE.
Solution: Either run your code with newer JRE or communicate with the author of the library containing "com.some.package.SomeClassName" whether there is a version for the older JRE you want to use.

Unable to start TestNG test via Maven surefire plugin

Sample message:

org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Caused by: org.apache.maven.surefire.testset.TestSetFailedException: Unknown TestNG version 6.9.13.6
at org.apache.maven.surefire.testng.TestNGExecutor.getConfigurator(TestNGExecutor.java:207)

Reason: Maven Surefire plugin must be at least version 2.19.1 in order to work with TestNG, which version is composed of more than three version tokens (example: 6.9.13.6)
Solution: Change the version of maven-surefire-plugin to latest like 3.0.0-M4 or newer in your pom.xml

...
<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>3.0.0-M4</version>
   ...
</plugin>
...

Log4j deadlock occurring between Logger and Appender

Symptom: The execution hangs and could not complete.

Reason: A deadlock happens when the Log4J Logger and the Appender try to log messages at the same time. Concrete reason might be DEBUG severity for ATS DB logging.

Solution:  Stop the logging with DEBUG level from the "org.apache.commons.dbcp2" package, so no deadlocks could happen.
Technical details: This is because of Log4J implementation details when Appender tries to emit another log message.
Example:

<logger name="org.apache.commons.dbcp2">
  <priority value="error" />
</logger>

Foreign key constraint exception on agent when executing monitoring operations

Sample message:

java.sql.SQLException: The INSERT statement conflicted with the FOREIGN KEY constraint "FK_tSystemStats_tStatsTypes". 
  The conflict occurred in database "SOME_DATABASE", table "dbo.tStatsTypes", column 'statsTypeId'.

Reason:  Agent works with old (non-existing anymore) monitoring statistics information.

Solution: Restart agent in order to clear statistic IDs cache.

Technical details: The first execution of monitoring operation on any agent caches statistic IDs in order to speed up inserting of system statistics. Those IDs are unique for each database.

If you change the ATS log database or recreate it and do not restart the agent, it will assume that the already cached IDs are still valid.

Cyclic dependencies when executing TestNG tests (6.14.2+)

Sample message:

The following methods have cyclic dependencies:
  FunctionalTest_TestNGEvents2.beforeSuite()[pri:0, instance:com.axway.ats.examples.FunctionalTest_TestNGEvents2@42538425]

Reason: [@Before/@After][Suite/Class] annotated method in base( parent) and child class have the same name.

Solution: Use different names for such methods in each class in the inheritance chain. Could be fixed in later TestNG versions.

TypeNotPresentException and NoClassDefFoudError when starting TestNG tests

Sample message:

java.lang.TypeNotPresentException: Type [unknown] not present
...
Caused by: java.lang.NoClassDefFoundError: org/testng/IInvokedMethodListener2

Reason: This is because of removed interface in newer versions of TestNG - 7.0 and higher. ATS framework versions 4.0.x have TestNG integration relying on this listener.

Solution: Use ATS 4.0.x versions with TestNG 6.x, preferably 6.10.

Next ATS versions will migrate to support newer TestNG versions.

Could not process XYZ event

Sample message:

*** ATS ***  2019-08-09 12:34:56.789 ERROR [QueueLoggerThread-Thread-1] autodb.DbEventRequestProcessor: Could not process 'com.axway.ats.log.autodb.events.InsertMessageEvent'
with message:
My message
Sender location:
null
Current event processor state:
RUN ID: 0,
SUITE ID: 0,
TESTCASE ID: -1

Or similar like

*** ATS ***  2019-08-09 12:34:38.668 ERROR [QueueLoggerThread-Thread-1] autodb.QueueLoggerThread: EXCEPTION
USER message:
Error running ADD_TESTCASE_METAINFO event
Message:
Cannot execute event AddTestcaseMetainfoEvent at this time expected state TEST_CASE_STARTED, actual state INITIALIZED

TYPE:
class com.axway.ats.log.autodb.exceptions.IncorrectProcessorStateException
CALL STACK:
com.axway.ats.log.autodb.exceptions.IncorrectProcessorStateException: Cannot execute event AddTestcaseMetainfoEvent at this time expected state TEST_CASE_STARTED, actual state INITIALIZED
at com.axway.ats.log.autodb.model.AbstractLoggingEvent.checkIfCanBeProcessed(AbstractLoggingEvent.java:156)
at com.axway.ats.log.autodb.DbEventRequestProcessor.processEventRequest(DbEventRequestProcessor.java:353)
at com.axway.ats.log.autodb.QueueLoggerThread.run(QueueLoggerThread.java:90)

Reason:

This indicates error state with ATS DB logging and usually does not affect test execution. The case is that message is trying to be logged into ATS DB but test state is not currently configured. This could happen when AtsDbLogger.getLogger(name).info(message) ( or another DB logging statement) is used within a static block or when ATS agent is not configured yet for the current test.

Solution:

• Do not use DB logging calls from static blocks: static { ... }. At that moment ATS does not know the test state yet. This is because testing runner (TestNG) had not notified ATS yet.
• If such error appears in Agent usually setting default Log4J severity higher than DEBUG (like INFO or WARN) could also help. Also it is preferred to use Log4J Logger.getLogger(class).info(message) instead of directly using AtsDbLogger.
• Start/stop monitoring only inside a method annotated with @Test. Currently (v4.0.7) it is not possible to reliably track test state and so start/stop monitoring within @Before/@After methods.
  Symptom in such case: DbEventRequestProcessor: Could not process 'com.axway.ats.log.autodb.events.InsertSystemStatisticEvent'

Also there are some exceptions that are "OK" if a certain operation is invoked on either the test executor or the ATS agent.

For example if you invoke ActionQueue.cancelQueue(java.lang.String) on the executor, in the agent log you will see either

• Could not process 'com.axway.ats.log.autodb.events.CleanupLoadQueueStateEvent'
• com.axway.ats.log.autodb.exceptions.NoSuchLoadQueueException

Those are "OK" exceptions, since you are explicitly cancelling a Load queue and ATS will not be able to find it later during the test execution.

Installation of TestExplorer  database fails for SQL Server

Sample message: Msg 15118, Level 16, State 1, Server XYZ, Line 1
Password validation failed. The password does not meet the operating system policy requirements because it is not complex enough.
...

Reason: Default password in ATS install scripts does not meet your DB server password complexity. Observed on SQL Server 2019 installation.
Solution: Currently, for ATS TestExplorer version 4.0.7, the user for the DB is not configurable since this is considered and internal (testing, non-production) tool. When needed for stronger security, you may edit the password and/or the user name in the install script.