[E2E] Adds staking, unstaking, and sending tests #653

dylanlott · 2023-04-05T17:55:45Z

Description

This PR introduces the E2E test harness and tests for Stake, Unstake, and Send commands.

The E2E test suite is run with the command make test_e2e which calls go test with the proper flags and arguments to run the Cucumber E2E tests.
The E2E tests work by loading a local Kubeconfig and connecting to the local cluster then issuing commands to a validator's client binary with the cluster's kubectl exec. This means that the test runner must have a LocalNet up and running with a properly configured Kubeconfig at the default $HOME/.kube/config path.
The tests stake validator001's address with 150000000001 POKT and then unstakes it and asserts no error was reported from either command.
After that, the tests send a balance of 150000000 POKT to validator002 and asserts no error was reported.

Issue

Fixes #466
Closes #589 (this PR achieves the same but adds more test cases)

Type of change

Please mark the relevant option(s):

List of changes

Adds the e2e/tests directory
Adds godog dependency for running tests.
Sets up the E2E test harness to build when -tags=e2e is provided.
Creates a Make target test_e2e that runs the E2E tests.
Defines the PocketClient single-function interface for interacting with a built binary.
Makes the Send command respect non_interactive flags when parsing passwords.
Adds a README and CHANGELOG for the e2e directory that contains documentation about how to use, develop with, and run the E2E tests.

Testing

make develop_test; if any code changes were made
Docker Compose LocalNet; if any major functionality was changed or introduced
k8s LocalNet; if any infrastructure or configuration changes were made

Staking & Unstaking test demo 🎥

Required Checklist

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have added, or updated, godoc format comments on touched members (see: tip.golang.org/doc/comment)
I have tested my changes using the available tooling
I have updated the corresponding CHANGELOG

If Applicable Checklist

I have updated the corresponding README(s); local and/or global
I have added tests that prove my fix is effective or that my feature works
I have added, or updated, mermaid.js diagrams in the corresponding README(s)
I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

* creates the e2e/tests directory for godog tests * adds a simple root command feature * adds a make target for running the e2e tests * adds godog to deps

- readme updates - adds mermaid chart to README - fixes rootDir in Tiltfile

- refactors out superfluous step defintion - cleanup

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

… THEY WERE

cr-gpt · 2023-04-05T17:55:49Z

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables for this repo. you could follow readme for more information

github-actions

The changelog validation failed with the following output:

Latest date in app/docs/CHANGELOG.md is incorrect.
Latest: 2023-03-31, Current: 2023-04-05

Latest date in build/docs/CHANGELOG.md is incorrect.
Latest: 2023-04-04, Current: 2023-04-05

Latest date in e2e/docs/CHANGELOG.md is incorrect.
Latest: 2023-03-31, Current: 2023-04-05

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

reviewpad · 2023-04-05T17:57:19Z

AI-Generated Pull Request Summary: This pull request introduces a new End-to-End (E2E) Testing Framework for Pocket 1.0 with a focus on behavior-driven development using Cucumber tests and Kubernetes orchestration. Various scenarios are added for validators, such as staking, unstaking, and sending POKT. The changes also include dependency updates, improvements in the passphrase input flow, and modifications to the CI/CD configurations. New documentation has been added, including a README and a CHANGELOG for the E2E framework. Lastly, there are updates to Tiltfile, Makefile, and the addition of new utility functions and binaries.

github-actions

The changelog validation failed with the following output:

Latest date in app/docs/CHANGELOG.md is incorrect.
Latest: 2023-03-31, Current: 2023-04-05

Latest date in build/docs/CHANGELOG.md is incorrect.
Latest: 2023-04-04, Current: 2023-04-05

Latest date in e2e/docs/CHANGELOG.md is incorrect.
Latest: 2023-03-31, Current: 2023-04-05

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

reviewpad · 2023-04-05T18:08:07Z

AI-Generated Pull Request Summary: This pull request introduces various updates and additions across multiple files, including the implementation of the Pocket 1.0 E2E (End-to-End) Testing Framework, new test files, and updated dependencies. Changes include adding new files like README.md, test files like validators.feature, and the documentation outlining the approach for E2E testing. There are also updates to existing files, such as modifications in .github/workflows/main.yml, account.go, and the Tiltfile. Additionally, new dependencies have been added to the go.mod file, and the CHANGELOG.md has been updated with various version changes.

cr-gpt · 2023-04-05T18:11:01Z

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables for this repo. you could follow readme for more information

github-actions

The changelog validation failed with the following output:

Latest date in app/client/doc/CHANGELOG.md is incorrect.
Latest: 2023-04-07, Current: 2023-04-10

Latest date in persistence/docs/CHANGELOG.md is incorrect.
Latest: 2023-04-07, Current: 2023-04-10

Latest date in rpc/doc/CHANGELOG.md is incorrect.
Latest: 2023-04-07, Current: 2023-04-10

Latest date in runtime/docs/CHANGELOG.md is incorrect.
Latest: 2023-04-07, Current: 2023-04-10

Latest date in shared/CHANGELOG.md is incorrect.
Latest: 2023-04-07, Current: 2023-04-10

Latest date in utility/doc/CHANGELOG.md is incorrect.
Latest: 2023-04-07, Current: 2023-04-10

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

reviewpad · 2023-04-10T16:21:35Z

AI-Generated Summary: This pull request introduces various changes across multiple files, focusing on improving governance parameters handling, adding and updating end-to-end tests, and modifying documentation. Key updates include adding conditional statements to enhance user experience, introducing new test files, modifying existing test functions, updating import statements and dependencies, and introducing new RPC functions and command-line interface capabilities. Additionally, changes have been made to the Makefile and the persistence package for better code organization and versatility.

cr-gpt · 2023-04-10T17:08:34Z

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables for this repo. you could follow readme for more information

The check succeeded, dismissing the review comment.

reviewpad · 2023-04-10T17:09:59Z

AI-Generated Summary: This pull request includes a wide array of changes across multiple files, with a focus on end-to-end testing, updating governance parameters, and dependency upgrades. Key changes involve the introduction of a new generic framework for managing parameters in the persistence/gov.go file, improved reading behavior of passphrase depending on a provided flag, and the addition of end-to-end testing using Godog in steps_init_test.go. Modifications were made to existing functions and new functions were added to improve the overall efficiency and maintainability of the code. The go.mod file includes several dependency updates and new additions, along with indirect dependencies. Several changes were made to test files, including the addition of new tests to improve coverage. Updated documentation and changelogs provide a better record of changes made throughout the project. Lastly, the E2E testing framework using Cucumber is established with a proposed solution and a detailed rollout plan. Overall, these changes enhance code quality, maintainability, and test coverage for the project.

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

cr-gpt · 2023-04-10T18:34:41Z

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables for this repo. you could follow readme for more information

reviewpad · 2023-04-10T18:36:18Z

AI-Generated Summary: This pull request introduces a new end-to-end (E2E) testing framework for the Pocket 1.0 project, with several new files added and modifications made to existing ones. Notable changes include the addition of a binary file tilt-button.png, new test files such as validators.feature and root.feature, and updates to dependency versions in the go.mod file. The framework is aimed at improving development practices and testing the core functionality of the Pocket binary against a LocalNet instance. Additionally, changes to the project's documentation and build workflows have been made, and a new test_e2e Makefile target has been introduced to facilitate E2E test execution. Developers should review these changes to ensure they don't impact the current codebase or introduce any unexpected behavior.

cr-gpt · 2023-04-10T18:52:31Z

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables for this repo. you could follow readme for more information

reviewpad · 2023-04-10T18:53:51Z

AI-Generated Summary: This pull request includes updates to the CHANGELOG.md file, adds e2e-tests documentation and a new CHANGELOG.md file to the e2e/docs directory, modifies the Makefile, and adds a new end-to-end test suite for a Pocket Network validator. Additionally, it updates the go.mod file with new dependencies related to the Cucumber testing framework, adds an Architecture Decision Record for an End-to-End Testing Framework, and modifies the .github/workflows/main.yml file to disable ARM64 builds. New test scenarios for the Root Namespace and Validators Namespace features are introduced, as well as a new E2E test framework documentation and a comprehensive guide for E2E testing setup. Binary file tilt-button.png is added to the e2e/docs directory, and conditional statements are added for controlling different user interaction scenarios.

Olshansk · 2023-04-10T20:44:04Z

e2e/tests/steps_init_test.go

@@ -211,7 +211,7 @@ func getClientset() (*kubernetes.Clientset, error) {
 	}
 	clientset, err := kubernetes.NewForConfig(kubeConfig)
 	if err != nil {
-		return nil, fmt.Errorf("failed to get clientset from config: %w", err)
+		return nil, fmt.Errorf("failed to12gg get clientset from config: %w", err)


@dylanlott Seems like a small typo merged in at the very end.

Olshansk · 2023-04-10T20:46:08Z

e2e/docs/E2E_FRAMEWORK.md

@@ -0,0 +1,229 @@
+# Pocket 1.0 E2E Testing Framework


@dylanlott It seems like a lot of changes made to this file were reverted.

Some of the headers are bolded again

Ommit toc is reverted

Dates were introduced back

All the nits and suggestions were added back

Please re-review when you have a chance

Addressed in #663

…p-modules * pokt/main: update pocket repo read.me (#667) Update reviewpad.yml [KEYBASE] Add improve comment on keybase config (#665) [E2E] Chore: Doc updates (#663) [E2E] Adds staking, unstaking, and sending tests (#653) [Utility][RPC][CLI] Querying governance parameters (Issue #619) (#622) [Persistence][Utility] Separate all CreateAndApply functions into more functional components - Issue #508 (#652) [Persistence][Utility] Pools Address hack removal + state accessor fix for params and flags (#654) [PERSISTENCE] SavePoints and Rollbacks design document (Issue #493) (#533) Update reviewpad.yml Added ChatGPT-CodeReview workflow (#649) Update reviewpad.yml Added default reviewpad.yml file (#648) [DevNet] tweaks for remote environments (#601) [Documentation] Swap validator and non-validator triggers when finished synching (#646) [Consensus] Configuration entry point state sync (#528)

dylanlott and others added 14 commits April 4, 2023 14:41

[E2E] adds initial E2E test skeleton

f1c83a4

* creates the e2e/tests directory for godog tests * adds a simple root command feature * adds a make target for running the e2e tests * adds godog to deps

[E2E] adds staking test

65e98e3

- readme updates - adds mermaid chart to README - fixes rootDir in Tiltfile

[E2E] adds unstaking test

644715f

- refactors out superfluous step defintion - cleanup

[CLI] makes Send command respect non-interactive mode

7bb9c84

[E2E] adds test for Send command

d90ac0c

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

commenting out build steps to speed up debugging PUT THESE BACK WHERE…

74872ea

… THEY WERE

attempting updated permissions for read and write

3fedbe1

fix

8cd8921

Update main.yml

1b46efb

Update main.yml

b681e85

Update main.yml

d6fa889

Update main.yml

06fe95c

Update main.yml

40dccbb

Update main.yml

fef6d6e

dylanlott added infra Core infrastructure - not protocol related tooling tooling to support development, testing et al testing Defining, adding, automating or modifying tests push-image Build and push a container image on non-main builds labels Apr 5, 2023

dylanlott added this to the M2: Pocket DoS (Devnet of Servicers) milestone Apr 5, 2023

dylanlott requested a review from okdas April 5, 2023 17:55

dylanlott self-assigned this Apr 5, 2023

reviewpad bot added the large label Apr 5, 2023

github-actions bot previously requested changes Apr 5, 2023

View reviewed changes

put tests back where we found them

3e2e041

github-actions bot previously requested changes Apr 5, 2023

View reviewed changes

adds back arm64 linux builds to main.yml build

c683760

github-actions bot previously requested changes Apr 10, 2023

View reviewed changes

dylanlott changed the base branch from main to e2e/harness April 10, 2023 17:04

dylanlott changed the base branch from e2e/harness to main April 10, 2023 17:04

dylanlott added 7 commits April 10, 2023 12:31

[E2E] adds test for Send command

c773c71

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

updates to uPOKT

d392a96

updates

4a18493

[E2E] adds test for Send command

cde72e6

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

[E2E] adds test for Send command

5e410d5

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

[E2E] adds test for Send command

aaf8747

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

[E2E] adds test for Send command

7c19ea5

- adds the Send test - makes the Send command respect non-interactive mode CLI flags - adds the E2E design doc and updates the README - adds the E2E changelog

dylanlott force-pushed the issue/466-cli-tests branch from 279d0bf to 7c19ea5 Compare April 10, 2023 18:34

fix

dfd58a4

dylanlott merged commit f4b9e50 into main Apr 10, 2023

dylanlott deleted the issue/466-cli-tests branch April 10, 2023 19:08

Olshansk reviewed Apr 10, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[E2E] Adds staking, unstaking, and sending tests #653

[E2E] Adds staking, unstaking, and sending tests #653

dylanlott commented Apr 5, 2023

cr-gpt bot commented Apr 5, 2023

github-actions bot left a comment

reviewpad bot commented Apr 5, 2023

github-actions bot left a comment

reviewpad bot commented Apr 5, 2023

cr-gpt bot commented Apr 5, 2023

github-actions bot left a comment

reviewpad bot commented Apr 10, 2023

cr-gpt bot commented Apr 10, 2023

reviewpad bot commented Apr 10, 2023

cr-gpt bot commented Apr 10, 2023

reviewpad bot commented Apr 10, 2023

cr-gpt bot commented Apr 10, 2023

reviewpad bot commented Apr 10, 2023

Olshansk Apr 10, 2023

Olshansk Apr 10, 2023

dylanlott Apr 10, 2023

[E2E] Adds staking, unstaking, and sending tests #653

[E2E] Adds staking, unstaking, and sending tests #653

Conversation

dylanlott commented Apr 5, 2023

Description

Issue

Type of change

List of changes

Testing

Required Checklist

If Applicable Checklist

cr-gpt bot commented Apr 5, 2023

github-actions bot left a comment

Choose a reason for hiding this comment

reviewpad bot commented Apr 5, 2023

github-actions bot left a comment

Choose a reason for hiding this comment

reviewpad bot commented Apr 5, 2023

cr-gpt bot commented Apr 5, 2023

github-actions bot left a comment

Choose a reason for hiding this comment

reviewpad bot commented Apr 10, 2023

cr-gpt bot commented Apr 10, 2023

reviewpad bot commented Apr 10, 2023

cr-gpt bot commented Apr 10, 2023

reviewpad bot commented Apr 10, 2023

cr-gpt bot commented Apr 10, 2023

reviewpad bot commented Apr 10, 2023

Olshansk Apr 10, 2023

Choose a reason for hiding this comment

Olshansk Apr 10, 2023

Choose a reason for hiding this comment

dylanlott Apr 10, 2023

Choose a reason for hiding this comment