ATS : Common test verifications

Introduction

A regular test scenario can be broken in three important parts or phases:

  1. Configuration - prepares the environment for the test execution
  2. Action - the actual execution of the test steps
  3. Verification - tells whether the product behaved as expected or not

We consider all three parts as very important. This page talks about the third one - the Verification



RBV introduction

We have noticed that some types of verification are needed for many of the test scenarios. In order to make these checks easier, we have come up with our Rule-Based Verification (or simply called RBV ) library

It allows you to easily do verifications on the following objects:

  • File and folders - using the FileSystemVerification class
  • Databases - using the DbVerification class
  • Mail in IMAP folders - using the ImapVerification class
  • Objects in S3 storage buckets - using S3Verification class
  • Your own custom objects



The RBV Engine

One of the main RBV features is that it has an internal engine which not only checks if the requested conditions are met, but it also re-tries as many times as needed.

Imagine you are expecting some file to appear at some remote server, but the file will probably not appear at the very first time you check. In such case, you need to check a few times before giving up.

RBV does all checks several times with a specific interval between the checks. You can change the polling configuration at run-time using the RbvConfigurator class. For example:

RbvConfigurator rbvConfigurator = RbvConfigurator.getInstance();

// number of maximum polls
rbvConfigurator.setPollingAttempts( 10 );

// you can put some initial delay before the first poll(in ms).
// Imagine you are sure the requested file will not appear before the first minute, 
// it does not make sense to keep polling prior the first minute is over
rbvConfigurator.setPollingInitialDelay( 60 * 1000 );

// the interval(in ms) between each poll
rbvConfigurator.setPollingInterval( 500 );

The above code will cause maximum 10 polling attempts with 0.5 seconds interval between every two attempts and the first poll will be executed after a minute has passed

RbvConfigurator keeps the default settings. But if needed, you can them overwrite for some particular instances.



File system verifications

Here is some very basic check on the file system

import com.axway.ats.rbv.clients.FileSystemVerification;

FileSystemVerification fsVerification = new FileSystemVerification( SERVER_IP, FILE_LOCATION );
fsVerification.verifyFileExists();

The above code will check whether a file exists on a remote host. If the file does not exist during the first poll, RBV will try over and over again until the file is found or until all attempts are exhausted.

If you want to work locally, use the constructor which does not specify a remote host IP

When working remotely, the ATS Agent must be running on the remote host



Database verifications

Here is some very basic check in a database

import com.axway.ats.rbv.clients.DbVerification;

TestBox serverBox = new TestBox();
serverBox.setHost( SERVER_IP );
serverBox.setDbType( "ORACLE" );
serverBox.setDbName( DATABASE_NAME );
serverBox.setDbUser( USER_NAME );
serverBox.setDbPass( USER_PASSWORD );

// make an instance providing the database credentials and the table to work with
DbVerification dbVerification = new DbVerification( serverBox, "Table1" );

// add one or more check rules
dbVerification.checkFieldValueContains( "", "column1", "How old" );

// trigger the verification process
dbVerification.verifyDbDataExists();

This example shows how we can check whether some cell value exists in the specified table. If the file does not exist during the first poll, RBV will try over and over again until the value is found or until all attempts are exhausted.



IMAP verifications

Here is some very basic check on mail in some IMAP folder

import com.axway.ats.rbv.clients.ImapVerification;

// define the connection and credential parameters for the IMAP folder of a particular user
ImapVerification imapVerification = new ImapVerification( SERVER_IP, "test1", "password" );

// check the subject
imapVerification.checkSubject( "RE: Hi" );

// check body contains some string
imapVerification.checkBody( "Financing of the transaction" );

// now trigger the process for finding the expected message
imapVerification.verifyMessageExists();

The above code will check whether the INBOX folder of some IMAP user contains a message with a specific subject and it contains some specific text in its body.



Remote storage (S3) verifications

Here is some very basic check on remote Amazon Simple Storage Service (S3) compatible services

import com.axway.ats.rbv.clients.S3Verification;

S3Verification s3Verification = new S3Verification( ENDPOINT, ACCESS_KEY, SECRET_KEY, BUCKET_NAME, BASE_DIR, OBJECT_NAME, true /* isRecursive */ );
s3Verification.checkMd5( "560bba529205da4d92bebb026b2e6e8b" );
s3Verification.verifyObjectExists();

The above code will check whether an object named OBJECT_NAME exists and is with given MD5 at some endpoint (remote host). If it does not exists during the first poll or the checksum does not match then RBV will try over and over until the object is found or until all attempts are exhausted.



Set configuration settings for the current verification instance

Using the following methods you can change configuration settings, just for the current verification instance.
These methods are accessible for File system, IMAP and Database verification instances.

method description
setPollingInitialDelay Set polling initial delay for the current instance only
setPollingAttempts Set polling attempts for the current instance only
setPollingInterval Set polling interval for the current instance only



Verification types

The following table looks somewhat complex, but it gives a very good picture of the behavior of RBV when using the different verify methods:


FileSystemVerification

DbVerification

ImapVerification

Description

verifyFileExists
verifyFolderExists

verifyDbDataExists

verifyMessageExists

Verify the searched object exists
Behavior: It will keep polling until the searched object is present and matches all the check rules.
So if for example, the object is present from the very beginning, the verification will succeed at the first poll otherwise, it will keep polling until the search object appears (ending with success) or the polling attempts are exhausted(ending with failure).

verifyFileDoesNotExist
verifyFolderDoesNotExist

verifyDbDataDoesNotExist

verifyMessageDoesNotExist

Verify the searched object does not exist
Behavior: It will keep polling until it verifies the searched object is not present.
So if for example, the object is not present from the very beginning, the verification will succeed at the first poll otherwise, it will keep polling until the search object disappear(ending with success) or the polling attempts are exhausted(ending with failure)

verifyFileAlwaysExists
verifyFolderAlwaysExists

verifyDbDataAlwaysExists

verifyMessageAlwaysExists

Verify the searched object exists all the time
Behavior: The searched object along with all rules for checking must be present for the whole polling period, this will result in success. If the searched object is not present even once, the verification fails right away.

verifyFileNeverExist
verifyFolderNeverExist

verifyDbDataNeverExists

verifyMessageNeverExists

Verify the searched object does not exist all the time
Behavior: The searched object must not be present for the whole polling period, this will result in success. If the searched object is present even once, the verification fails right away.





Back to parent page

Go to Table of Contents