Introduction

ATS comes with a library called Environment Cleanup which takes care of environment backups and restores, but usually the preferred way is to use this library silently i.e. without writing Java code but rather just describe it in some XML file. And this page shows how to achieve it.

Describing the environment

We have explained how to deal with creating, deploying and running Agent components here. As ATS Agent is component based, it is recommended that each Agent component describes its own environment.

For example, if you have one Agent component which is used for user management and you have another used for file transfers, you will probably want to describe different database tables for both Agent components.

Now we will jump directly into a somewhat complex environment description. If you remember, each Agent component has one file called agent_descriptor.xml like the following file:

<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE component SYSTEM "agent_descriptor.xsd">
<component name="myproduct_actions">

    <actionClass name="com.myproduct.actions.SimpleActions"/>
    <actionClass name="com.myproduct.actions.FileTransferActions"/>

    <cleanupHandler name="com.myproduct.actions.SimpleCleanupHandler"/>

</component>

Now lets extend this file to the following:

<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE component SYSTEM "agent_descriptor.xsd">
<component name="myproduct_actions">

    <actionClass name="com.myproduct.actions.SimpleActions"/>
    <actionClass name="com.myproduct.actions.FileTransferActions"/>

    <cleanupHandler name="com.myproduct.actions.SimpleCleanupHandler"/>

    <!--Environment description-->
    <environment backupFolder="/tmp/component_environment_backup" windowsBackupFolder="C:/component_environment_backup" >

        <!-- important DB tables -->
        <database host="10.10.10.10" port="33060" name="myproduct_db" user="root" password="123" type="mysql">
	    	<table name="Banks"/>
	    	<table name="Users" autoIncrementResetValue="0"/>
	    	<table name="Partners" dropTable="true"/>
	    	<table name="PartnerCerts" lock="false"/>
	    	<table name="BusinessUnit" columnsToExclude="data"/>
        </database>

	    <!-- important files -->
        <file path="../../../file1"/>
        <file path="../../file2" backupName="file2_backup">
	    	<action command="/etc/init.d/cmd1" sleep="2"/>
        </file>
        <file path="/tmp/file3" backupName="file3_backup"/>
        <file path="/tmp/file4">
	    	<action command="/etc/init.d/cmd1" sleep="3"/>
	    	<action command="/etc/init.d/cmd2" sleep="1"/>
	    </file>

	    <!-- important directories -->
        <directory path="/usr/dir1/" backupName="dir1_backup"/>
        <directory path="/tmp/dir2/"/>
        <directory path="/tmp/dir3/">
	    	<action command="/etc/init.d/cmd3" sleep="2"/>
	    </directory>
    </environment>

</component>

This example is somewhat complex, but it shows a lot. Here are the important details:

environment node - It describes the environment of our Agent component.
It has 2 backup folders for Unix and Windows which define the folder where we will backup the environment into.
If both folder attributes are not defined - the system temporary folder will be used instead.
This folder will be read later when doing environment restore.
database node - it describes the database we want to maintain. You have to specify all the DB connection parameters and the DB type. Db type value could be: ORACLE, MYSQL, MSSQL, CASSANDRA and POSTGRESQL (since v4.0.7). Character case is not important.

database table nodes - the tables listed here will be the ones to backup/restore. Some options are possible here:
- columnToExclude specifies to skip some table columns. This is handy if some columns have too much but not important content
- lock specifies whether we will lock the table while working with it
- dropTable specifies whether table will be dropped and recreated after that. This might be used when the table size is very big and the "delete" query is slow.
- autoIncrementResetValue requests to reset the auto-increment value. You should not use it unless needed
file nodes and directory nodes - specify files or whole directories that are to be backed up/restored. You can use full or relative paths.
If for some reason you want to specify the name of the backup file/directory - use the backupName attribute. For example you will need to use it when backing up files with same names coming from different locations.
actions assigned to some file or directory node. Sometimes after the restoration of some file or directory you will need to execute some shell command(this is usually used to restart some system process). This option gives you the chance to execute the needed shell command. The sleep attribute specifies how long time to wait after the execution of your shell command. For example if you have to restart a JBoss application server you may need to wait 60 seconds and then go on.

Important considerations

Databases

Oracle database

The foreign keys constraints should be configured with DEFERRABLE in order to allow database restore.

This does not break the database security. It is used to allow temporally disabling of foreign keys checks till the end of the database restore transaction. More information is available at this resource.

PostgreSQL database

The user performing DB restore should have superuser role. The requirement comes from the need to enable/disable all triggers and it is similar to the Oracle DB case. When designing your test automation you should use just the DB restore with such power user and do the rest of DB automation with regular user. More info for superuser role needed for disable/enable triggers is here: https://www.postgresql.org/docs/current/sql-altertable.html

Files and directories

Files backup

If a backup file already exists - we skip the backup;

If the file to backup does not exist - no backup is made.

While backing up, the last modification date from the original file is set to the backup file as well.

Files restore

Original file will be restored if the file to restore is missing or it exists and its last modification date is different than the one of the backup file.

If the file to restore exists but there is no backup file, we will actually delete this file.

Directory backup

This directory is backed up recursively including all its files and sub-directories including the empty ones.

If the backup directory does not exist, it will be created. When the backup operation is done, we will have a backup copy identical to the original directory.

Directory restore

The directory restore operation brings the directory of interest back to its original state including all its sub-directories and empty directories

Additional actions on file and directory restore

You can have many additional actions on same file/directory restore. This is the case when more than one process is affected by same file/directory.
You can have the same shell command on more than one file/directory. This is the case when same process is affected by more than one restore file/directory.

In order to shorten the time it takes to run all additional actions, we follow the next sequence:

Restore all files and directories one by one
Execute all shell commands one by one. If a shell command is specified more than once - it is executed just once.
Sleep as long as the longest sleep interval is specified

So In the given example will execute just once /etc/init.d/cmd1, /etc/init.d/cmd2 and etc/init.d/cmd3. Then we will sleep for 3 seconds.

More than one database in same environment

We support the maintenance of more than one database.
Just add another database node to use this feature.

More than one environment

Sometimes it is useful to have described more than one environment. Here is an example:

    <environment name="environment1"  backupFolder="/tmp/component_environment_backup" windowsBackupFolder="C:/component_environment_backup" >
        <!-- list all the items of this environment -->
    </environment>

    <environment name="environment2"  backupFolder="/tmp/component_environment_backup2" windowsBackupFolder="C:/component_environment_backup2" >
        <!-- list all the items of this environment -->
    </environment>

Note the usage of the name attribute of each environment, it is needed so can distinguish them on restore. It is also needed to specify different backup folders of all environments.

By default we will create a backуп of all described environments, but you will have to specify which one to restore later.

Custom restore steps

There is a way to define some custom steps which will be automatically called after environment restore. These steps you need to define into a so called cleanup handler.

If interested, see the "Creating a cleanup handler" section here

Backing up the environment

This is really easy. As you have described your environment(s) in the agent_descriptor.xml, all you need to do is to simply package and deploy it with your Agent component.

When the Agent is started, it will read the agent_descriptor.xml file, extract all the environment related information and create the backups as described. If you have many environments - they are all backed-up in their backup folders.

But in case you want to create a backup at some specific moment or if you want to specify which exact environment to be backed up, then make an instance of EnvironmentCleanupClient and use some of its methods. Here is one example:

// create an instance of this client, by providing the ATS Agent address
EnvironmentCleanupClient cleanupClient = new EnvironmentCleanupClient( "10.10.10.10" );

// backup the specified environment of the specified component
cleanupClient.backup( "myproduct_actions", "environment1" );

Restoring the environment

You need to run Java code like the following:

// create an instance of this client, by providing the ATS Agent address
EnvironmentCleanupClient cleanupClient = new EnvironmentCleanupClient( "10.10.10.10" );

// run a restore command for the specified component and environment name
cleanupClient.restore( "myproduct_actions", "environment1" );

Right at this moment your environment will be restored as described in the agent_descriptor.xml file.

Check the other methods in this class for more options.

Back to parent page

Go to Table of Contents

ATS : Environment maintenance using ATS Agent