Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Geo-Spatial Plugin integration test #3244

Merged

Conversation

andy-k-improving
Copy link
Contributor

@andy-k-improving andy-k-improving commented Jan 13, 2025

Description

This is a PR which aim to introduce a new IT test to assert and make sure the existing project setup and logic will work when Geo-Spatial installed within the same cluster.

High-level code changes include:

  • Update Gradle file to download Geo-Spatial plugin for Integration test.
  • Add an new test-case to perform assertion during PPL Integ-test time to make sure Plugin is available, the actual tests against GeoIp functionality will be added in sub-sequent PR along with the business logic.

Test-plan:

  • Integ-testIT passed

Related Issues

Resolves: #3037

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@LantaoJin
Copy link
Member

Could you update the code base and re-run the CI workflows again?

YANG-DB
YANG-DB previously approved these changes Jan 14, 2025
@andy-k-improving
Copy link
Contributor Author

Checked the log from CI build, it's blocked by #3235.

YANG-DB
YANG-DB previously approved these changes Jan 15, 2025
@andy-k-improving andy-k-improving force-pushed the ft-ak-geo-spatial-integration branch from efe579f to cf83890 Compare January 15, 2025 18:24
@andy-k-improving
Copy link
Contributor Author

@LantaoJin All tests are passing except bwc, can you have another look?
Thanks!

@@ -261,6 +297,11 @@ testClusters {
plugin(getJobSchedulerPlugin())
plugin ":opensearch-sql-plugin"
}
integTestWithGeo {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason for separating integTestWithGeo from integTest?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

integTestWithGeo got separated as it requires an additional plugin Geo-spatial, by having a separate build target with external deps will help troubleshot in the case of build failure.
Similar to the existing build target: integTestWithSecurity.

Copy link
Collaborator

@penghuo penghuo Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by having a separate build target with external deps will help troubleshot in the case of build failure.

could u elberate more on this. i think integTest already have clear hint on failed test cases.

If Geo command/function is native PPL feature, and OpenSearch release by defalut included geo-spatial plugin It not necessary to sepreate it from integTest. The geo-spatial use case is more similar to job-scheduler, instead of security plugin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@penghuo Thanks for your comment, actually you do have the point on this, especially geo-spatial being packaged with OpenSearch release.
If that is the case, I will remove this CI and move the test-case back to ppl package.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the code to reflect above, @penghuo can you have another look?
Thanks!

Copy link
Collaborator

@acarbonetto acarbonetto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andy-k-improving
Copy link
Contributor Author

@YANG-DB

Signed-off-by: Andy Kwok <andy.kwok@improving.com>

Remove redundent wrapper

Signed-off-by: Andy Kwok <andy.kwok@improving.com>

Github CI

Signed-off-by: Andy Kwok <andy.kwok@improving.com>

Update spotless

Signed-off-by: Andy Kwok <andy.kwok@improving.com>

Update gradle def

Signed-off-by: Andy Kwok <andy.kwok@improving.com>

Exclude geo IT from main integ-test

Signed-off-by: Andy Kwok <andy.kwok@improving.com>
Signed-off-by: Andy Kwok <andy.kwok@improving.com>
Signed-off-by: Andy Kwok <andy.kwok@improving.com>
Signed-off-by: Andy Kwok <andy.kwok@improving.com>
@andy-k-improving andy-k-improving force-pushed the ft-ak-geo-spatial-integration branch from c54639c to e9a4273 Compare January 20, 2025 19:06
@penghuo penghuo merged commit ef3d7b6 into opensearch-project:main Jan 21, 2025
16 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.x
# Create a new branch
git switch --create backport/backport-3244-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ef3d7b6606285daa4de86f793a6647b93e6a9000
# Push it to GitHub
git push --set-upstream origin backport/backport-3244-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-3244-to-2.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x backport-failed infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE]Add iplocation function to PPL for IP address geolocation
5 participants