Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Initial Codebase for Integrations CLI #1488

Closed
wants to merge 41 commits into from

Conversation

Swiddis
Copy link
Collaborator

@Swiddis Swiddis commented Apr 3, 2023

Description

Per issue #1451, this PR implements the basic initial setup for the integrations CLI. From the help page:

Usage: integrations-cli.py [OPTIONS] COMMAND [ARGS]...

  Create and maintain Integrations for the OpenSearch Integrations Plugin.  

Options:
  --help  Show this message and exit.

Commands:
  check    Analyze the current Integration and report errors.
  create   Create a new Integration from a specified template.
  package  Package the current integration for use in OpenSearch.

It has some robustness to be added after the API is ready, particularly with regard to generating and validating sub-components such as saved objects. The current implementation only generates a configuration and references the local schema files.

The PR also creates a CI workflow that tests the new Python code.

TODO before converting from draft:

  • Documentation
  • More robust error handling
  • General code cleanup

Issues Resolved

Part of progress on #1451. This issue can safely be archived and superseded by smaller-scope issues.
Conflicts with #1454.

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Poetry is an alternative package manager for Python that is more
scalable. It's used in various large Python projects to good
effect. https://python-poetry.org/

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
@Swiddis Swiddis added enhancement New feature or request ci labels Apr 3, 2023
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
@YANG-DB YANG-DB added this to the 2.8 milestone Apr 3, 2023
YANG-DB
YANG-DB previously approved these changes Apr 3, 2023
Copy link
Member

@YANG-DB YANG-DB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a README.md file at the root of the integration-cli package

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
@Swiddis
Copy link
Collaborator Author

Swiddis commented Apr 12, 2023

Updated TODO list:

  • General Code Clean-up
    • Work on reworking usage of returns (first experiment was sloppy)
    • Fix up error handling
    • Break commands into smaller units for testing
  • Error handling in clear non-happy-path cases
  • Fix the CI pipeline
  • Add more testing
  • Switch to Pytest for consistency with other OS CLI tools
  • Convert to a proper Python package
  • Actual dependency management and validation
    • Detect when components contain other components and add them
    • Validate if subcomponents are missing
  • Improve catalog searching functionality
  • Proper automatic integration uploading

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
This reverts commit 1a178f3.
It starts work on new functionality when the functionality depends on other stuff.
Revert to a clean version.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants