Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: integration tests fixes + library improvements #1177

Merged
merged 14 commits into from
Dec 18, 2024
Merged

feat: integration tests fixes + library improvements #1177

merged 14 commits into from
Dec 18, 2024

Conversation

jzvikart
Copy link
Contributor

Risks

Very low. Worst case this could break the tests or introduce problems with dependencies.

Background

What does this PR do?

This builds on top of previous changes that introduced the first version of integration tests framework. These changes:

  • fix some existing issues with smoke and integration tests failing (esp. giving agent a fixed time to start that was not always sufficient)
  • extend integration test library with a full wrapper for setting up / tearing down a test
  • refactor existing integration test ("Hello Trump") to use new library
  • fix a potential issue with possible leak of API keys (not related to integration tests themselves)
  • remove a dependency that was previously added but is no longer required

What kind of change is this?

Improvement + bug fix + feature

Why are we doing this? Any context or related work?

This is to improve overall project quality via better testing..

Documentation changes needed?

None

Testing

To test the tests, these changes need to be run in CI workflow.
If either smoke or integration tests fail, the PR should NOT be merged. In that case we will check the logs and update the PR as necessary.

Deploy Notes

None

Database changes

None

Deployment instructions

None

Discord username

user98634

@pgoos
Copy link
Contributor

pgoos commented Dec 18, 2024

@lalalune @shakkernerd how is it possible that integration-tests (pull_request) don't have access to OPENAI_API_KEY where integration-tests (pull_request_target) DO have such access. See a failure https://github.com/ai16z/eliza/actions/runs/12389865584/job/34583749108?pr=1177#step:7:14
How was the secret configured?

@jzvikart jzvikart marked this pull request as ready for review December 18, 2024 15:49
@shakkernerd
Copy link
Member

@lalalune @shakkernerd how is it possible that integration-tests (pull_request) don't have access to OPENAI_API_KEY where integration-tests (pull_request_target) DO have such access. See a failure https://github.com/ai16z/eliza/actions/runs/12389865584/job/34583749108?pr=1177#step:7:14 How was the secret configured?

I am going to have to look into that.

@shakkernerd shakkernerd merged commit 1781ae1 into elizaOS:develop Dec 18, 2024
4 of 6 checks passed
AbdelStark added a commit to AbdelStark/eliza that referenced this pull request Dec 18, 2024
@AbdelStark
Squashed commit of the following:
edc39e2 
commit 4e27c73
Merge: d1bdf00 a01157f
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:21:56 2024 -0800

    Merge pull request elizaOS#1209 from marcNY/patch-1

    docs: Update README.md

commit a01157f
Merge: 502d4a1 d1bdf00
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:20:33 2024 -0800

    Merge branch 'develop' into patch-1

commit d1bdf00
Merge: b3ebb3c eecb77f
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:18:45 2024 -0800

    Merge pull request elizaOS#1207 from v1xingyue/v1xingyue-patch-1

    fix: gitpod cicd bug

commit b3ebb3c
Merge: 1781ae1 7b9c285
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:08:17 2024 -0800

    Merge pull request elizaOS#1201 from 9547/doc-cn

    docs(cn): add python 3.7

commit 502d4a1
Author: Marc F <8562443+marcNY@users.noreply.github.com>
Date:   Wed Dec 18 16:44:46 2024 +0000

    Update README.md

    Fix starter script: Add missing ‘cd’ command to navigate to ‘eliza-starter’ directory

commit 1781ae1
Merge: 855c029 be74341
Author: Shakker Nerd <165377636+shakkernerd@users.noreply.github.com>
Date:   Wed Dec 18 16:43:48 2024 +0000

    Merge pull request elizaOS#1177 from Sifchain/jure/integration-tests

    feat: integration tests fixes + library improvements

commit eecb77f
Author: v1xingyue <qixingyue@gmail.com>
Date:   Wed Dec 18 22:46:20 2024 +0800

    gitpod cicd bug

    Sometimes we can't fetch tags, so let's fetch tags before.

commit be74341
Merge: 969db65 855c029
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Wed Dec 18 10:15:08 2024 +0100

    Merge remote-tracking branch 'ai16z/develop' into jure/integration-tests

commit 7b9c285
Author: 9547 <29431502+9547@users.noreply.github.com>
Date:   Wed Dec 18 16:59:26 2024 +0800

    docs(cn): rm duplicated -

    Signed-off-by: 9547 <29431502+9547@users.noreply.github.com>

commit 3405828
Author: 9547 <29431502+9547@users.noreply.github.com>
Date:   Wed Dec 18 16:58:30 2024 +0800

    docs(cn): add python2.7

    Signed-off-by: 9547 <29431502+9547@users.noreply.github.com>

commit 969db65
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 13:16:52 2024 +0100

    Change CI trigger to 'pull_request'

commit c558ad6
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:54:40 2024 +0100

    Disable logging of fetch requests to avoid disclosing sensitive information

commit 6d24565
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:14:31 2024 +0100

    Minor cleanup

commit e90263f
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:09:04 2024 +0100

    Remove dependency zx

commit 9888ced
Merge: 3fab472 0a23d6d
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:03:45 2024 +0100

    Merge remote-tracking branch 'ai16z/develop' into realitySpiral/integration-tests

commit 3fab472
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 19:59:33 2024 +0100

    Avoid using console.clear() to preserve messages for debugging

commit e5fb5d5
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 19:53:08 2024 +0100

    Cleanup

commit 11eff66
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 19:47:10 2024 +0100

    Get stopAgent() to work as expected

commit a439407
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 15:56:26 2024 +0100

    Cleanup

commit 87f862f
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 15:42:36 2024 +0100

    Fixes

commit 9b0302c
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 15:25:28 2024 +0100

    Wait for server to start

commit 6424c39
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 14:04:50 2024 +0100

    Refactoring

commit 8824041
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 13:50:09 2024 +0100

    Increase delay to avoid CI/CD timeouts
AbdelStark added a commit to AbdelStark/eliza that referenced this pull request Dec 18, 2024
@AbdelStark
Squashed commit of the following:
1ce6614 
commit 4e27c73
Merge: d1bdf00 a01157f
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:21:56 2024 -0800

    Merge pull request elizaOS#1209 from marcNY/patch-1

    docs: Update README.md

commit a01157f
Merge: 502d4a1 d1bdf00
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:20:33 2024 -0800

    Merge branch 'develop' into patch-1

commit d1bdf00
Merge: b3ebb3c eecb77f
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:18:45 2024 -0800

    Merge pull request elizaOS#1207 from v1xingyue/v1xingyue-patch-1

    fix: gitpod cicd bug

commit b3ebb3c
Merge: 1781ae1 7b9c285
Author: Odilitime <janesmith@airmail.cc>
Date:   Wed Dec 18 09:08:17 2024 -0800

    Merge pull request elizaOS#1201 from 9547/doc-cn

    docs(cn): add python 3.7

commit 502d4a1
Author: Marc F <8562443+marcNY@users.noreply.github.com>
Date:   Wed Dec 18 16:44:46 2024 +0000

    Update README.md

    Fix starter script: Add missing ‘cd’ command to navigate to ‘eliza-starter’ directory

commit 1781ae1
Merge: 855c029 be74341
Author: Shakker Nerd <165377636+shakkernerd@users.noreply.github.com>
Date:   Wed Dec 18 16:43:48 2024 +0000

    Merge pull request elizaOS#1177 from Sifchain/jure/integration-tests

    feat: integration tests fixes + library improvements

commit eecb77f
Author: v1xingyue <qixingyue@gmail.com>
Date:   Wed Dec 18 22:46:20 2024 +0800

    gitpod cicd bug

    Sometimes we can't fetch tags, so let's fetch tags before.

commit be74341
Merge: 969db65 855c029
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Wed Dec 18 10:15:08 2024 +0100

    Merge remote-tracking branch 'ai16z/develop' into jure/integration-tests

commit 7b9c285
Author: 9547 <29431502+9547@users.noreply.github.com>
Date:   Wed Dec 18 16:59:26 2024 +0800

    docs(cn): rm duplicated -

    Signed-off-by: 9547 <29431502+9547@users.noreply.github.com>

commit 3405828
Author: 9547 <29431502+9547@users.noreply.github.com>
Date:   Wed Dec 18 16:58:30 2024 +0800

    docs(cn): add python2.7

    Signed-off-by: 9547 <29431502+9547@users.noreply.github.com>

commit 969db65
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 13:16:52 2024 +0100

    Change CI trigger to 'pull_request'

commit c558ad6
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:54:40 2024 +0100

    Disable logging of fetch requests to avoid disclosing sensitive information

commit 6d24565
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:14:31 2024 +0100

    Minor cleanup

commit e90263f
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:09:04 2024 +0100

    Remove dependency zx

commit 9888ced
Merge: 3fab472 0a23d6d
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Tue Dec 17 04:03:45 2024 +0100

    Merge remote-tracking branch 'ai16z/develop' into realitySpiral/integration-tests

commit 3fab472
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 19:59:33 2024 +0100

    Avoid using console.clear() to preserve messages for debugging

commit e5fb5d5
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 19:53:08 2024 +0100

    Cleanup

commit 11eff66
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 19:47:10 2024 +0100

    Get stopAgent() to work as expected

commit a439407
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 15:56:26 2024 +0100

    Cleanup

commit 87f862f
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 15:42:36 2024 +0100

    Fixes

commit 9b0302c
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 15:25:28 2024 +0100

    Wait for server to start

commit 6424c39
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 14:04:50 2024 +0100

    Refactoring

commit 8824041
Author: Jure Žvikart <7929905+jzvikart@users.noreply.github.com>
Date:   Mon Dec 16 13:50:09 2024 +0100

    Increase delay to avoid CI/CD timeouts
@odilitime odilitime mentioned this pull request Dec 21, 2024
Merged
@jzvikart jzvikart mentioned this pull request Dec 27, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@monilpat monilpat monilpat approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jzvikart @pgoos @shakkernerd @monilpat