Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update CodeGen Xeon and Gaudi Kubernetes codegen.yaml and docs #595

Closed
wants to merge 61 commits into from
Closed

Update CodeGen Xeon and Gaudi Kubernetes codegen.yaml and docs #595

wants to merge 61 commits into from

Conversation

dmsuehir
Copy link
Contributor

Description

This PR has a few updates based on issues that I ran into when deploying the CodeGen example on a cluster for xeon and Gaudi. The following issues are addressed in the PR:

  • I added a note about potentially using a persistent volume claim instead of having to create the /mnt/opea-models directory on the nodes
  • Deploying the codegen.yaml files gave an error like: 
    error: error validating "codegen.yaml": error validating data: [unknown object type "nil" in ConfigMap.data.http_proxy, unknown object type "nil" in ConfigMap.data.https_proxy, unknown object type "nil" in ConfigMap.data.no_proxy]; if you choose to ignore these errors, turn validation off with --validate=false
    This error is because the ConfigMap in the yaml has a few env vars that are just empty (nil). Changing these to have empty quotes "" fixes the issue.
  • I added a note about it taking a couple of minutes for the service to start and how to check the logs, because I ran into an issue where the curl command failed like "curl: (18) transfer closed with outstanding read data remaining" and it was just because the service wasn't ready yet. Also, knowing how to check the logs is useful for watching the status and figuring out if the curl command is failing because of an error.
  • When running on Gaudi wasn't working for me ("RuntimeError: synStatus=26 [Generic failure] Device acquire failed.") until I added the hugepages-2Mi/memory to the resource limits. The habana documentation for Kubernetes shows it using hugepages-2Mi and memory in the resources, so that seems to be the recommended config.

Issues

N/A

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

N/A

Tests

Manually tested the changes on a Kubernetes cluster with Xeon (GNR) and Gaudi 2 nodes.

@dmsuehir
Fix the CodeGen Kubernetes ConfigMap 'nil' proxy var error
883c28a 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Updates after running on Gaudi to add hugepages-2Mi/memory resources
ab57d04 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Fix typo
08c0f97 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Removing blank line
4ccae79 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
ashahba
ashahba approved these changes Aug 14, 2024
View reviewed changes
Copy link
Collaborator

@ashahba ashahba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@chensuyue @dmsuehir
Enhance CI/CD infrastructure (opea-project#593)
39dda6a 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
@chensuyue @dmsuehir
Fix left issues in CI/CD structure refactor (opea-project#599)
33d2753 
Signed-off-by: chensuyue <suyue.chen@intel.com>
@chensuyue @dmsuehir
Add BoM collect workflow and image publish workflow (opea-project#600)
d3b4e08 
Signed-off-by: chensuyue <suyue.chen@intel.com>
@chensuyue @dmsuehir
set action back to pull_request_target (opea-project#602)
a934a28 
Signed-off-by: chensuyue <suyue.chen@intel.com>
dmsuehir and others added 18 commits August 15, 2024 11:33
@dmsuehir
Merge branch 'main' into dina/codegen_proxy_empty
a9c7834
@dmsuehir
Merge branch 'main' into dina/codegen_proxy_empty
9071f7e
@dmsuehir
Fix the CodeGen Kubernetes ConfigMap 'nil' proxy var error
b10b760 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Updates after running on Gaudi to add hugepages-2Mi/memory resources
808d96d 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Fix typo
3c1cfec 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Removing blank line
2c6fa68 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Enhance CI/CD infrastructure (opea-project#593)
a160348 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Fix left issues in CI/CD structure refactor (opea-project#599)
dbcb80b 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Add BoM collect workflow and image publish workflow (opea-project#600)
99d93bf 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
set action back to pull_request_target (opea-project#602)
a2c8014 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Enhance CI/CD infrastructure (opea-project#593)
56c5460 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
set action back to pull_request_target (opea-project#602)
712963e 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Fix CD workflow typos (opea-project#605)
eef07d3 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@zhlsunshine @dmsuehir
Add AudioQnA example via GMC (opea-project#597)
a978e0f 
* add AudioQnA example via GMC.
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>

* add more information for e2e test scritpts.
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>

* fix bug in e2e test scripts.
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
remove continue-on-error: true to stop the test when image build fail…
899cf1f 
…ed (opea-project#608)

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@letonghan @dmsuehir
update port for dataprep in set_env.sh (opea-project#606)
e46f6ac 
Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@XuehaoSun @pre-commit-ci @dmsuehir
Freeze base images (opea-project#607)
55e0be3 
* Update Dockerfile to use LANGCHAIN_VERSION argument

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>

* Revert "Update Dockerfile to use LANGCHAIN_VERSION argument"

This reverts commit 1bff239.

* chore: Add manual freeze images workflow

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* split jobs

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>

---------

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@daisy-ycguo @dmsuehir
Add gmc build, scan and deploy workflow (opea-project#611)
9b4c803 
Signed-off-by: Yingchun Guo <yingchun.guo@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
zhlsunshine and others added 28 commits August 16, 2024 09:34
@zhlsunshine @dmsuehir
Add dataprep microservice to chatQnA example and the e2e test (opea-p…
f7b591d 
…roject#589)

Signed-off-by: zhlsunshine <huailong.zhang@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@jaswanth8888 @pre-commit-ci @dmsuehir
fixed ISSUE-528 (opea-project#590)
9616208 
* fixed ISSUE-528

Signed-off-by: jaswanth8888 <karani.jaswanth@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: jaswanth8888 <karani.jaswanth@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@daisy-ycguo @dmsuehir
add composible manifest e2e for cd workflow (opea-project#588)
5b02384 
Signed-off-by: Yingchun Guo <yingchun.guo@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@XinyaoWa @chensuyue @dmsuehir
Fix vLLM and vLLM-on-Ray UT bug (opea-project#580)
1ecee49 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@letonghan @dmsuehir
Fix ChatQnA Qdrant CI issues (opea-project#569)
660e401 
Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Fix the CodeGen Kubernetes ConfigMap 'nil' proxy var error
4f83274 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Updates after running on Gaudi to add hugepages-2Mi/memory resources
218c95e 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Fix typo
a99f7e6 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Removing blank line
4006b7d 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Enhance CI/CD infrastructure (opea-project#593)
9c45ee5 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Fix left issues in CI/CD structure refactor (opea-project#599)
cfe98e8 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Add BoM collect workflow and image publish workflow (opea-project#600)
220b0cf 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
set action back to pull_request_target (opea-project#602)
1b2dfa7 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Enhance CI/CD infrastructure (opea-project#593)
1cde81e 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
set action back to pull_request_target (opea-project#602)
82e3594 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Fix CD workflow typos (opea-project#605)
8383b87 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@zhlsunshine @dmsuehir
Add AudioQnA example via GMC (opea-project#597)
8323f15 
* add AudioQnA example via GMC.
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>

* add more information for e2e test scritpts.
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>

* fix bug in e2e test scripts.
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
remove continue-on-error: true to stop the test when image build fail…
90e1aba 
…ed (opea-project#608)

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@letonghan @dmsuehir
update port for dataprep in set_env.sh (opea-project#606)
0cbcc34 
Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@XuehaoSun @pre-commit-ci @dmsuehir
Freeze base images (opea-project#607)
22370fb 
* Update Dockerfile to use LANGCHAIN_VERSION argument

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>

* Revert "Update Dockerfile to use LANGCHAIN_VERSION argument"

This reverts commit 1bff239.

* chore: Add manual freeze images workflow

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* split jobs

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>

---------

Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@daisy-ycguo @dmsuehir
Add gmc build, scan and deploy workflow (opea-project#611)
c522326 
Signed-off-by: Yingchun Guo <yingchun.guo@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@daisy-ycguo @dmsuehir
change microservice tags in CD workflow (opea-project#612)
31c9fe6 
Signed-off-by: Yingchun Guo <yingchun.guo@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Fix the CodeGen Kubernetes ConfigMap 'nil' proxy var error
c432f72 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Removing blank line
e4cf14a 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
Enhance CI/CD infrastructure (opea-project#593)
e2c899d 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@chensuyue @dmsuehir
set action back to pull_request_target (opea-project#602)
b4bdace 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Merge branch 'dina/codegen_proxy_empty' of github.com:dmsuehir/GenAIE…
b2b5710 
…xamples into dina/codegen_proxy_empty

Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Resolve conflict
171c473 
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir
Copy link
Contributor Author

Reopened as PR #613 because rebasing seems to have gone wrong after trying to fix the DCO check

@dmsuehir dmsuehir closed this Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ashahba ashahba ashahba approved these changes

@lvliang-intel lvliang-intel Awaiting requested review from lvliang-intel lvliang-intel is a code owner

@letonghan letonghan Awaiting requested review from letonghan

@Spycsh Spycsh Awaiting requested review from Spycsh

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.