This repository has been archived by the owner on Sep 8, 2024. It is now read-only.
Voight Kampff test suite #2506
Merged
Voight Kampff test suite #2506
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
devs-mycroft
added
the
CLA: Yes
forslund
force-pushed
the
feature/voight-kampff-pr
branch
from
2daa651
to
7adefb3
Compare
|
shouldn't the "meta" field simply be a part of message.context ? |
|
It could definitely be moved there, I did consider it but felt like the context should be more "standardized" and not depend to much on the message type. Like the skill shouldn't interfere with the context and let the "surrounding machinery" handle that field. Not sure if that makes sense? I don't really have a strong opinion one way or another, but that was my thoughts at the time. |
forslund
force-pushed
the
feature/voight-kampff-pr
branch
from
7adefb3
to
1fcb961
Compare
chrisveilleux
suggested changes
Mar 21, 2020
test/integrationtests/voight_kampff/features/steps/utterance_responses.py
Outdated
Show resolved
Hide resolved
test/integrationtests/voight_kampff/features/steps/utterance_responses.py
Outdated
Show resolved
Hide resolved
test/integrationtests/voight_kampff/features/steps/utterance_responses.py
Outdated
Show resolved
Hide resolved
test/integrationtests/voight_kampff/features/steps/utterance_responses.py
Outdated
Show resolved
Hide resolved
forslund
force-pushed
the
feature/voight-kampff-pr
branch
2 times, most recently
from
a3ae425
to
75b5a6b
Compare
The Voight kampff test is an integration test collecting and running behave test of skills.
Some branches have a "/" in their name (e.g. feature/new-and-cool) Some commands, such as those tha deal with directories, don't play nice with this naming convention. Define an alias for the branch name that can be used in these scenarios.
Behave will generate test results in the allure format, this will be picked up by Jenkins and sent of to a standalone webserver.
Docker build will now perform most actions of the dev-setup making it possible to use caches in a greater extent speeding up the build
This reduces the verbosity during the operation. This is not of much interest, mainly success or failure is what matters.
Allure commandline and chown command are now run through the Jenkinsfile instead of through the run_test_suite.sh
Provides meta data such as dialog used and data that was inserted.
Override the bus clients on_message method and collect all messages in a list. This will make it harder to miss a message during a test
Will now print all responses received from Mycroft
- Sharing only the identity file removes the need for clearing the skill sandbox dir and padatious cache - Make things a bit cleaner with separate Allure volume
forslund
force-pushed
the
feature/voight-kampff-pr
branch
from
75b5a6b
to
fe6d2f5
Compare
|
I think I've addressed all concerns here, either with a comment or changed code. |
chrisveilleux
approved these changes
Mar 23, 2020
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| main(sys.argv[1:]) |
There was a problem hiding this comment.
you don't need to pass the command line arguments to the arg parser. it looks at sys.argv by default.
There was a problem hiding this comment.
I personally like the traceability. main get's an argument to operate on so all actions can be traced back to it without any questions of where the settings originate.
krisgesling
reviewed
Mar 24, 2020
- Pulseaudio is launched if on CI - remove hard-coding of start-mycroft.sh
This will make sure that the isSpeaking flag is cleared correctly
Set USER and CI
The backend doesn't generate any audio playback, only consumes the messages
This switches behave to the current dev release including the autoretry system.
--url can be added to specify the repo url --branch can be added to specify a specific branch
forslund
force-pushed
the
feature/voight-kampff-pr
branch
from
fe6d2f5
to
9fe158d
Compare
Takes in arguments for both test_setup.py and behave test runner. Parses any args for test_setup and passes any remaining arguments to behave. This moves argparsing out of the test_setup main() allowing the helper commands to pass in pre-parsed arguments rather than adding logic inside main to differentiate between a list and a preparsed arument object
1 task
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add the voight_kampff test suite.
This allows creating behavioral tests for skills using the behave framework. It specifies a couple of pre-defined Given, When and Then covering the most common test cases but can be expanded with custom "step files".
In addition of being a test suite for skill creators, it will be used as an integration test validating pull requests to mycroft-core, making sure the essential skills work.
There are very little changes to core, but a couple of modifications are included in all these commits:
Contributor license agreement signed?
CLA [ Yes ]