AWS – Device Farms Integration

Creating access keys for execution on AWS

In order to create access keys for execution on AWS, follow the steps given below:

1. Log in to the AWS console successfully.

2. In the navigation bar on the upper right, click your user name, and then choose My Security Credentials.

To navigate to ‘My Security Credentials’

3. Click on the ‘Access keys’ section.

Access keys section

4. Click on the ‘Create New Access Key’ button to generate the access keys.

Click to generate access keys

5. Choose ‘Show Access Key’ to view it, then copy the keys to somewhere else for safekeeping. OR

6. Choose ‘Download Key File’ to save the access key ID and secret access key to a .csv file at a secure location on your computer.

AWS Configuration in Avo Assure

In the client package provided for Avo Assure, the following changes should be made to enable execution of mobile test scenarios on AWS device farm:

1. Change only the values for the keys mentioned below in AWS_config.json file which resides at – ‘AvoAssure\assets\AWS_assets’:

   1. "projectname": <<existing_project_name>>

   2. "poolname": <<existing_device_pool_of_the_project_specified>>

Update the key-value pairs specified in AWS_config.json

2. Open the credentials file as a text file in any text editor from the following location – ‘AvoAssure\assets\AWS_assets\config’. Enter the acquired access keys as values for the respective keys.

Update the credentials file with valid access key values

Triggering AWS execution by Avo Assure

1. Log in as a valid Avo Assure user who has been assigned projects of application type - ‘Mobile App’.

2. Navigate to any execution task which is of ‘Mobile App’ type.

Note:

• For a successful execution on AWS, make sure that the test steps within the selected execution task are AWS compatible.

• This can be done by debugging each test case within the selected execution task.

• By debugging the test cases involved, the user is notified (if there are any test steps that are not AWS compatible) in the ICE logs.

• The user can then, comment on the incompatible test steps and execute the task on AWS for a successful run.

3. Click the ‘AWS’ icon from the ‘Execute On’ options.

Click and highlight the ‘AWS’ icon

4. Click on the ‘Execute’ button. An overlay will be displayed with the following message on the Avo Assure web tool:

Avo Assure web tool when execution is in progress.

5. The following details will be displayed on ICE during the AWS execution:

AWS execution progress logs in ICE.

6. Once, the execution is completed/terminated, the user is prompted with a dialog box to go to the ‘Reports’ plugin.

7. After execution completes, logs similar to the following are displayed on ICE:

Detailed logs in ICE on AWS execution completion

Validating result of run on AWS

1. Log in to the AWS console successfully.

2. Navigate to ‘Device Farm’ service.

3. Click on the project name in which the execution was run (which was specified in the AWS_config.json file).

4. The user will be able to see the execution entry in the ‘Automated tests’ tab with an auto-generated name (with date-time of run) specified for the run.

Execution entry of run in AWS device farm

5. Click the execution entry.

6. Select the device of which the test execution needs to be validated.

7. After selecting the device, the user can go through the various results generated by AWS in sections Video, Suites, Test specification log, Logs, Performance, Files, and Screenshots.

Results generated for specific device on AWS execution

Note: Refer to the logs section for a detailed test steps report.

Validating result of run in Avo Assure Client

After successful AWS execution of mobile test scenarios using Avo Assure, the results are downloaded for the Avo Assure user in the client package for ease of access.

1. After AWS execution completes, an ‘output’ folder is created in the ‘AvoAssure\assets\AWS_assets’ location.

2. This ‘output’ folder will consist of downloaded files (Customer Artifacts.zip, Test spec output.txt, and Video.mp4) from AWS. Also, the user is notified about the download, in the ICE console.

Files downloaded in ‘output’ folder from AWS Device Farm

Limitations

1. Usage of dynamic variables across test steps is not implemented.

2. As AWS takes only one application per run, the keyword – ‘launchApplication’ is considered at its first occurrence in the test suite. All subsequent ‘launchApplication’ keywords in the test suite are ignored.

3. Only two Generic keywords are supported currently- createDynVariable and wait. The implementation of other Generic Keywords to be supported on AWS is a work in progress.

4. The following keywords which belong to mobile scripting are not AWS compliant:

verifyDate	GetMultipleViewsByIndexes	GetValue
closeApplication	GetSelectedViews	SetValue
getDevices	GetViewByIndex	GetSlideValue
VerifyTime	SelectMultipleViewsByIndexes	SetSlideValue
installApplication	SelectMultipleViewsByText	CellClick
invokeDevice	SelectViewByIndex	GetCellValue
stopServer	SelectViewByText	GetRowCount
uninstallApplication	VerifyAllViews	VerifyCellValue
VerifyRowCount	VerifyListCount	SetNumber
GetAllViews	VerifySelectedViews	GetListCount

5. Integration of AWS reports on Avo Assure web ‘Reports’ plugin is a work in progress.