Skip to content

Latest commit

 

History

History
215 lines (153 loc) · 11 KB

top-extensions-node-js-test-framework.md

File metadata and controls

215 lines (153 loc) · 11 KB
Raw

Overview

MsPortalFx-Test is an end-to-end test framework that is written in TypeScript abd runs tests against the Microsoft Azure Portal. It tests extension interactions with user behavior, moreso than extension interactions with the Portal. Its open source contribution model focuses on partner needs instead of internal Portal needs. It is distributed independently from the SDK to allow developers to develop tests in the same language as the extension.

The test framework interacts with the Portal as a user would, and helps developers make performant and robust extensions when they decrease breaking changes to partner team CI.

General Architecture

MsPortalFx-Test is arranged into three layers of abstraction. The names of the layers may not be consistent in all instances, but the general idea is the same. There may also be some future refactoring to easily differentiate between the layers.

  1. Controls layer

    The controls layer contains the basic controls used in the Portal, like blades, checkboxes, textboxes, and others. It is not used directly by tests in most cases, because it is mainly for composing the action layer and the test layers. It is built on webdriver primitives and there is little or no retry logic.

  2. Actions layer

    The actions layer performs an action and verifies whether it was successful. It can retry the action if necessary. It is used in writing tests like portal.openBrowseBlade. It is built upon the controls layer.

  3. Test layer

    The test layer contains wrappers that are used for testing common functionality. It is built upon the action and control layers, and is used in writing business-case tests or other forms of suite-test functionality, like parts.canPinAllBladeParts. It can re-run the test if necessary, and throws an exception if the test or verification fails.

Download

MsPortalFx-Test is available for download on npm under the package name msportalfx-test https://aka.ms/portalfx/msportalfx-test.

Getting Started

  1. Install Node.js. (last tested on 8.11.3 LTS)

  2. Install TypeScript 1.8.10 or greater. TypeScript 2.3.3 is located at http://download.microsoft.com/download/6/D/8/6D8381B0-03C1-4BD2-AE65-30FF0A4C62DA/2.3.3-TS-release-dev14update3-20170519.1/TypeScript_Dev14Full.exe.

  3. Open a command prompt and create a directory for your test files.

    md e2etests

  4. Switch to the new directory and install the msportalfx-test module by using npm:

    cd e2etests
npm install msportalfx-test --no-optional

  5. The msportalfx-test module comes with useful TypeScript definitions and its dependencies. To make them available to your tests, we recommend that you use the typings Typescript Definition Manager.

  6. First install the typings Typescript Definition Manager globally:

    npm install typings -g

  7. Next copy the provided typings.json and package.json files from the node_modules/msportalfx-test/typescript folder (there are other files there as well but you can ignore those) to your test root directory and use typings to install the provided definitions.

     *copy typings.json to your test root directory*
 *navigate to your test root directory*
 typings install

  8. In the package.json file, you update the chromedriver and msportalfx-test versions to their latest versions found at

    https://www.npmjs.com/package/chromedriver

    https://www.npmjs.com/package/msportalfx-test

  9. The package.json describes your project and its dependencies for NPM to understand. You can find more details about it at https://docs.npmjs.com/files/package.json

  10. Once the dependencies have been updated, install them by running the following command:

    npm install --no-optional

  11. Now you are ready to write a test.

Write a test

For this test example you'll need an existing cloud service for the test you'll write below, so if you don't have one please go to the Azure Portal and create a new cloud service. Write down the dns name of your cloud service to use it in your test.

We'll use the Mocha testing framework to layout the following test, but you could use any framework that supports Node.js modules and promises. Mocha is included by default in the package.json file you copied earlier.

Create a portaltests.ts file in your e2etests directory and paste the following.

/// <reference path="./typings/index.d.ts" />
 
import assert = require('assert');
import testFx = require('MsPortalFx-Test');
import nconf = require('nconf');
import until = testFx.until;
 
describe('Cloud Service Tests', function () {
    this.timeout(0);
    
    it('Can Browse To A Cloud Service', () => {
 
        
        // Load command line arguments, environment variables and config.json into nconf
        nconf.argv()
            .env()
            .file(__dirname + "/config.json");
 
        //provide windows credential manager as a fallback to the above three
        nconf[testFx.Utils.NConfWindowsCredentialManager.ProviderName] = testFx.Utils.NConfWindowsCredentialManager;
        nconf.use(testFx.Utils.NConfWindowsCredentialManager.ProviderName);
        
        
        testFx.portal.portalContext.signInEmail = 'johndoe@outlook.com';
        testFx.portal.portalContext.signInPassword = nconf.get('msportalfx-test/johndoe@outlook.com/signInPassword');
        
        // Update this variable to use the dns name of your actual cloud service 
        let dnsName = "mycloudservice";
        
        return testFx.portal.openBrowseBlade('microsoft.classicCompute', 'domainNames', "Cloud services (classic)").then((blade) => {
            return blade.filterItems(dnsName);
        }).then((blade) => {
            return testFx.portal.wait<testFx.Controls.GridRow>(() => {
                return blade.grid.rows.count().then((count) => {
                    return count === 1 ? blade.grid.rows.first() : null;
                });
            }, null, "Expected only one row matching '" + dnsName + "'.");
        }).then((row) => {
            return row.click();			
        }).then(() => {            
            let summaryBlade = testFx.portal.blade({ title: dnsName + " - Production" });
            return testFx.portal.wait(until.isPresent(summaryBlade));
        }).then(() => {
            return testFx.portal.quit();
        });
    });
});

Remember to replace "mycloudservice" with the dns name of your actual cloud service. Also replace the signInEmail and password values with your test account credentials

NOTE: @microsoft.com accounts are not supported.

In this test we start by importing the MsPortalFx-Test module. Then the credentials are specified for the user that will sign in to the Portal. These should be the credentials of a user that already has an active Azure subscription.

After that we can use the Portal object to drive a test scenario that opens the Cloud Services Browse blade, filters the list of cloud services, checks that the grid has only one row after the filter, selects the only row and waits for the correct blade to open. Finally, the call to quit() closes the browser.

Add the configuration

Create a file named config.json next to portaltests.ts. Paste this in the file.

	{
	"capabilities": {
		"browserName": "chrome"
	},
	"portalUrl": "https://portal.azure.com"
	}

This configuration tells MsPortalFx-Test that Google Chrome should be used for the test session and https://portal.azure.com should be the Portal under test.

Compile and run

Compile your TypeScript test file:

tsc portaltests.ts --module commonjs

and then run Mocha against the generated JavaScript file.

NOTE: Using an elevated command prompt may cause Chrome to crash or hang. If so, try using a non-elevated command prompt:

npm test

The following output will be sent to your console as the test progresses:

    Portal Tests
Opening the Browse blade for the microsoft.classicCompute/domainNames resource type...
Starting the ChromeDriver process...
Performing SignIn...
Waiting for the Portal...
Waiting for the splash screen to go away...
Applying filter 'mycloudservice'...
    √ Can Browse To A Cloud Service (37822ms)	

    1 passing (38s)

If you run into a compilation error with node.d.ts, verify that the tsc version you are using is 1.8.x or newer. You can check the version by running:

tsc --version

If the version is incorrect, then you may need to adjust your path variables or directly call the correct version of tsc.exe.

More Documentation and Examples

The full documentation is available on the npm site https://aka.ms/portalfx/msportalfx-test.

Additional examples are available in the source code located at https://github.com/Azure/msportalfx-test/tree/master/test.

If you do not have access to the Microsoft Azure github organization and you are part of Microsoft, you can join by following the instructions at http://aka.ms/azuregithub.

Contributing

Contributions to improve the test framework are encouraged. If you develop a feature/fix a bug/etc that you feel would be useful to other users of the test framework then please submit a pull request to the Github repository located at https://aka.ms/portalfx/msportalfx-test. In order to access the source code and submit a pull request, you need to be part of the Microsoft Azure Github organization. You can check by using the site located at http://aka.ms/azuregithub.

For detailed instructions, please view the full documentation on the npm site located at https://aka.ms/portalfx/msportalfx-test.