GitHub Actions for Yandex Cloud Load Testing service

USAGE EXAMPLE

USAGE EXAMPLE WORKFLOW

NOTE: All actions require auth-key-json to authorize requests to Yandex Cloud services. Please check this documentation page for instructions how to create one.

Short reference

• test-suite - execute multiple load tests sequentially
• agents-create - create and start load testing agents using Yandex Cloud Compute VMs as a hosting (Compute is billed separately)
• agents-delete - delete load testing agents
• test-single-run - execute a single load test
• test-single-check - check results of a single load test

Action: test-suite

uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main

Execute and assess a bunch of tests in a single action step.

Each test is specified by a separate directory containing:

• test-config.yaml test configuration file
• (optional) meta.json json file specifying test properties
• (optional) additional test data files which will be transferred to the agent via data-bucket

A list of directories containing test-config.yaml test configuration files should be provided via test-directories input parameter.

NOTE: Why data-bucket is needed? Test data files often contain sensitive information such as access tokens and customer data. Using data-bucket as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

Inputs

Input	Description	Default	Required
folder-id	ID of a folder in Yandex Cloud.	n/a	yes
auth-key-json-base64	BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.	n/a	no
auth-key-json	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
action-log-level	Action log level.	NOTICE	no
data-bucket	Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent. WARNING: CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket WARNING2: agent service account must be able to download data from this bucket.	``	no
artifacts-bucket	Bucket to store artifacts generated by agent during test execution. WARNING: agent service account must be able to upload data to this bucket.	``	no
test-directories	A whitespace-separated list of directory paths containing test configuration files.	n/a	yes
agent-filter	A filter expression to select agents to execute tests. Example: - 'name contains github' - agents with 'github' substring in name. - 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID	labels.workflow=${{ github.run_id }}	no
add-description	Specified string will be added to description of created tests.	Run from GitHub Actions	no
add-labels	Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'.	ci=github	no

Outputs

Output	Description
test-ids	IDs of performed tests.
test-infos-file	File containing a JSON array of objects with information about performed tests.
test-infos	JSON array of objects with information about performed tests.
execution-report-file	Path to generated .md execution report file.
checks-report-file	Path to generated .md checks report file.
artifacts-dir	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

• loadtesting.loadTester - to create and run the test
• storage.uploader - to upload test data to Object Storage (required, if data-bucket is specified)

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directories: |-
      _impl/testdata/http-const-rps
      _impl/testdata/https-const-rps

Action: agents-create

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

Create agents alongside with Cloud Compute VMs to host them.

This action is usually used used in pair with 'agents-delete' action.

NOTE: For security reasons you should create new service account with 'loadtesting.generatorClient' role to assign to loadtesting agent (see service-account-id input below)

Inputs

Input	Description	Default	Required
folder-id	ID of a folder in Yandex Cloud.	n/a	yes
auth-key-json-base64	BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.	n/a	no
auth-key-json	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
action-log-level	Action log level.	NOTICE	no
count	Number of agents to be created.	1	no
service-account-id	ID of a service account to create agent VM with. The service account must have 'loadtesting.generatorClient' role.	``	no
vm-zone	Compute zone to create agent VM in.	ru-central1-d	no
name-prefix	If count=0, a name of created agent. if count>0, a name prefix of created agents.	onetime-ci-agent	no
description	Description of created agents.	Create from Github Actions	no
labels	Labels of created agents. Format: 'key1=value1,key2=value2'.	workflow=${{ github.run_id }}	no
cli-args	Additional command line arguments to be passed to yc loadtesting agent create (see yc loadtesting agent create -h for more details). Override rules: * '--zone ARG' overrides 'vm-zone' input parameter * '--service-account-id ARG' overrides 'service-account-id' input parameter * '--network-interface' overrides default network interface settings (which is - 1-to-1 NAT (dynamic public IP) in automatically chosen subnet) * '--name ARG' overrides 'name-prefix' input parameter * '--description ARG' overrides 'description' input parameter * '--labels' overrides 'labels' input parameter	``	no
timeout	Time to wait for agents to become READY_FOR_TEST. Usually, reaching this timeout means either missing permissions or invalid agent network settings.	600	no

Outputs

Output	Description
agent-ids	IDs of created agents.
artifacts-dir	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

• loadtesting.loadTester - to create an agent
• compute.editor - to create and start agent VM
• iam.serviceAccounts.user - to assign service account to agent VM
• vpc.user - to configure network interface on agent VM
• vpc.publicAdmin - to configure public IP on agent VM (the default way, may be omitted if you know what you are doing)

Examples

Simple:

# - 2 CPU, 2 GB RAM
# - one-to-one NAT in an automatically selected subnet

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
  labels: "workflow=${{ github.run_id }}"
  # count: 1
  # vm-zone: ru-central1-a

Custom CPU and RAM settings:

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
  labels: "workflow=${{ github.run_id }}"
  cli-args: |-
    --cores 2
    --memory 2G
  # count: 1
  # vm-zone: ru-central1-a

Custom network settings:

This version is essentially identical to yc loadtesting agent create ${cli-args}.

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  cli-args: |-
    --service-account-id "${{ vars.YC_LOADTESTING_SA_ID }}"
    --labels "workflow=${{ github.run_id }}"
    --cores 2
    --memory 2G
    --zone 'ru-central1-a'
    --network-settings "subnet-name=default-a,security-group-ids=${{ vars.YC_LOADTESTING_AGENT_SECURITY_GROUP_ID }}"
  # count: 1

Action: agents-delete

uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main

Delete agents (and the VMs).

This action is usually used used in pair with 'agents-create' action.

Inputs

Input	Description	Default	Required
folder-id	ID of a folder in Yandex Cloud.	n/a	yes
auth-key-json-base64	BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.	n/a	no
auth-key-json	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
action-log-level	Action log level.	NOTICE	no
agent-ids	A string containing whitespace-separated list of agents to be deleted. Normally, should be used with conjunction with agents-create.outputs.agent-ids.	n/a	yes

Outputs

Output	Description
artifacts-dir	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

• loadtesting.loadTester - to delete an agent
• compute.editor - to stop and delete agent VM

Example

This action is usually used in pair with agents-create action:

loadtesting:
  name: loadtesting job
  runs-on: ubuntu-latest
  steps:
    - id: create-agents
      uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
      with:
        folder-id: ${{ vars.YC_FOLDER_ID }}
        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
        service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
    #
    # here we run some tests on created agents
    # - id: run-test
    #   ...
    #
    - id: delete-agents
      if: always() # make sure delete is called even if previous steps fail
      uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main
      with:
        folder-id: ${{ vars.YC_FOLDER_ID }}
        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

        # pass created agent ids to make sure they are deleted
        agent-ids: ${{ steps.agent-create.outputs.agent-ids }}

Action: test-single-run

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-run@main

Execute a single test.

Each test should be specified by a separate directory containing:

• test-config.yaml test configuration file
• (optional) meta.json json file specifying test properties
• (optional) additional test data files which will be transferred to the agent via data-bucket

A directory containing test-config.yaml test configuration file should be provided via test-directory input parameter.

NOTE: Why data-bucket is needed? Test data files often contain sensitive information such as access tokens and customer data. Using data-bucket as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

Inputs

Input	Description	Default	Required
folder-id	ID of a folder in Yandex Cloud.	n/a	yes
auth-key-json-base64	BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.	n/a	no
auth-key-json	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
action-log-level	Action log level.	NOTICE	no
data-bucket	Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent. WARNING: CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket WARNING2: agent service account must be able to download data from this bucket.	``	no
artifacts-bucket	Bucket to store artifacts generated by agent during test execution. WARNING: agent service account must be able to upload data to this bucket.	``	no
test-directory	A path to a directory containing test configuration files.	n/a	yes
agent-filter	A filter expression to select agents to execute tests. Example: - 'name contains github' - agents with 'github' substring in name. - 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID	labels.workflow=${{ github.run_id }}	no
add-description	Specified string will be added to description of created tests.	Run from GitHub Actions	no
add-labels	Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'.	ci=github	no

Outputs

Output	Description
test-id	ID of performed test.
test-info-file	File containing a JSON object with information about performed test.
test-info	JSON object with information about performed test.
report-file	Path to generated .md report file.
artifacts-dir	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

• loadtesting.loadTester - to create and run the test
• storage.uploader - to upload test data to Object Storage (required, if data-bucket is specified)

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directory: _impl/testdata/https-const-rps

Action: test-single-check

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-check@main

Assess results measured in a single test run.

The assessment is performed via custom scripts (check_summary.sh and check_report.sh) which can be added to test directory:

TEST_ID="bb4asdfjljbhh" # some test id
TEST_DIR="_impl/testdata/http-const-rps" # test-directory

yc --format json loadtesting test get $TEST_ID > summary.json
yc --format json loadtesting test get-report-tables $TEST_ID > report.json

set -e
$DIR/check_summary.sh summary.json
$DIR/check_report.sh report.json

If check_report.sh or check_summary.sh are not found in test directory, some basic checks will be performed instead.

Inputs

Input	Description	Default	Required
folder-id	ID of a folder in Yandex Cloud.	n/a	yes
auth-key-json-base64	BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.	n/a	no
auth-key-json	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
action-log-level	Action log level.	NOTICE	no
test-id	ID of a test to be checked.	n/a	yes
test-directory	Directory with test configuration files.	n/a	yes

Outputs

Output	Description
raw-output	Raw stdout of a check script.
report-file	Path to generated .md report file.
artifacts-dir	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

• loadtesting.loadTester - to retrieve information about the test

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main
  id: run-test
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directory: _impl/testdata/https-const-rps
- uses: yandex-cloud/yc-github-loadtesting-ci/test-check-single@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-id: ${{ steps.run-test.outputs.test-id }}
    test-directory: _impl/testdata/https-const-rps