Skip to content

Commit af3abd4

Browse files
manavgupmanavgup
authored
feat: Add Secure IBM Cloud Code Engine Deployment (#405)
* feat: Add secure IBM Cloud Code Engine deployment workflow - Creates a Dockerfile.codeengine for production builds. - Adds a deploy_codeengine.sh script to automate deployment. - Replaces the insecure .env.codeengine file with a secure GitHub Actions workflow using repository secrets. - The new workflow can be manually triggered to deploy the application to IBM Cloud Code Engine. * fix: Address all PR feedback for Code Engine deployment - Rewrites Dockerfile.codeengine to match project standards, fixing Python version, dependencies, security, and file paths. - Simplifies deploy_codeengine.sh to only handle deployment, not image building. - Overhauls GitHub Actions workflow to manage the build/push lifecycle correctly and pass all required environment variables. * fix: Address critical and high-priority feedback for deployment - Dockerfile: Corrects Python healthcheck, Poetry version, Rust command, PYTHONPATH, and adds build caching. - Script: Fixes typo and adds resource limits. - Workflow: Adds security permissions and uses variables for configuration. * feat: Complete IBM Cloud Code Engine deployment with infrastructure - Add complete application deployment workflow (deploy_complete_app.yml) - Deploy infrastructure: PostgreSQL, MinIO, etcd, Milvus - Deploy applications: Backend + Frontend with proper service URLs - Add daily builds at 2 AM UTC with optional deployment - Include comprehensive security scanning with Trivy - Add smoke tests for all components - Update documentation for complete deployment strategy - Fix command injection vulnerability in deployment scripts - Align Dockerfile with project standards Resolves all critical issues from PR #405 reviews * fix: Address critical PR review issues - Fix PyTorch version mismatch (2.7.1 -> 2.5.0, 0.22.1 -> 0.20.0) - Remove .env.act file completely (workflow uses GitHub secrets) - Fix URL extraction in smoke tests (.status.url instead of .status.latest_ready_revision_name) - Add project selection to deployment scripts - Add .env.act to .gitignore for security Resolves critical issues from PR #405 review * fix: Use correct project name 'rag-modulo' instead of 'rag-modulo-test-project' - Update deploy_codeengine.sh to use 'rag-modulo' project name - Update deploy_infrastructure_codeengine.sh to use 'rag-modulo' project name - Remove test project name references * refactor: Move deployment scripts to .github/scripts/ for better CI/CD architecture - Move scripts from scripts/ to .github/scripts/ to clearly indicate CI/CD purpose - Rename scripts for clarity: deploy-backend.sh, deploy-frontend.sh, deploy-infrastructure.sh - Update workflow paths to use new script locations - Improve script documentation with clear variable requirements and usage examples This addresses the architectural concern about having deployment scripts in the general scripts/ folder when they are primarily used for CI/CD operations. * feat: rename deployment scripts to include codeengine suffix - Rename deploy-backend.sh to deploy-backend-codeengine.sh - Rename deploy-infrastructure.sh to deploy-infrastructure-codeengine.sh - Rename deploy-frontend.sh to deploy-frontend-codeengine.sh - Update GitHub Actions workflow to use new script names - Add comprehensive IBM MCP Context Forge style documentation - Improve script headers with detailed usage examples and security features This improves clarity by making it explicit that these scripts are specifically for IBM Cloud Code Engine deployment.
1 parent 24f7867 commit af3abd4

13 files changed

+2073
-0
lines changed

‎.coverage.Manavs-MacBook-Pro.local.36389.XNSeFZax‎

-92 KB
Binary file not shown.

‎.github/workflows/deploy_code_engine.yml‎

Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
name: Deploy to IBM Cloud Code Engine
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
skip_security_scan:
7+
description: 'Skip security scanning (not recommended)'
8+
required: false
9+
default: false
10+
type: boolean
11+
12+
# Define environment variables for the entire workflow
13+
env:
14+
APP_NAME: ${{ vars.IBM_CE_APP_NAME || 'rag-modulo-app' }}
15+
IBM_CLOUD_REGION: ${{ vars.IBM_CLOUD_REGION || 'us-south' }}
16+
CR_NAMESPACE: ${{ vars.IBM_CR_NAMESPACE || 'rag_modulo' }}
17+
18+
# Prevent concurrent deployments
19+
concurrency:
20+
group: deploy-code-engine-${{ github.ref }}
21+
cancel-in-progress: false
22+
23+
jobs:
24+
build-and-push-image:
25+
runs-on: ubuntu-latest
26+
permissions:
27+
contents: read
28+
packages: write
29+
security-events: write
30+
steps:
31+
- name: Check out code
32+
uses: actions/checkout@v4
33+
34+
- name: Set up Docker Buildx
35+
uses: docker/setup-buildx-action@v3
36+
37+
- name: Log in to IBM Cloud Container Registry
38+
uses: docker/login-action@v3
39+
with:
40+
registry: ${{ env.IBM_CLOUD_REGION }}.icr.io
41+
username: iamapikey
42+
password: ${{ secrets.IBM_CLOUD_API_KEY }}
43+
44+
- name: Build and push Docker image
45+
uses: docker/build-push-action@v5
46+
with:
47+
context: .
48+
file: ./Dockerfile.codeengine
49+
push: true
50+
tags: ${{ env.IBM_CLOUD_REGION }}.icr.io/${{ env.CR_NAMESPACE }}/${{ env.APP_NAME }}:${{ github.sha }}
51+
cache-from: type=gha
52+
cache-to: type=gha,mode=max
53+
54+
security-scan:
55+
needs: build-and-push-image
56+
if: ${{ !inputs.skip_security_scan }}
57+
runs-on: ubuntu-latest
58+
permissions:
59+
contents: read
60+
security-events: write
61+
steps:
62+
- name: Check out code
63+
uses: actions/checkout@v4
64+
65+
- name: Run Trivy vulnerability scanner
66+
uses: aquasecurity/trivy-action@master
67+
with:
68+
image-ref: ${{ env.IBM_CLOUD_REGION }}.icr.io/${{ env.CR_NAMESPACE }}/${{ env.APP_NAME }}:${{ github.sha }}
69+
format: 'sarif'
70+
output: 'trivy-results.sarif'
71+
72+
- name: Upload Trivy scan results to GitHub Security tab
73+
uses: github/codeql-action/upload-sarif@v3
74+
if: always()
75+
with:
76+
sarif_file: 'trivy-results.sarif'
77+
78+
- name: Run Trivy vulnerability scanner (table format)
79+
uses: aquasecurity/trivy-action@master
80+
with:
81+
image-ref: ${{ env.IBM_CLOUD_REGION }}.icr.io/${{ env.CR_NAMESPACE }}/${{ env.APP_NAME }}:${{ github.sha }}
82+
format: 'table'
83+
exit-code: '1'
84+
severity: 'CRITICAL,HIGH'
85+
86+
deploy-to-code-engine:
87+
needs: [build-and-push-image, security-scan]
88+
if: always() && (needs.security-scan.result == 'success' || needs.security-scan.result == 'skipped')
89+
runs-on: ubuntu-latest
90+
steps:
91+
- name: Check out code
92+
uses: actions/checkout@v4
93+
94+
- name: Set up IBM Cloud CLI
95+
uses: ibm-cloud/sdk-action@v1
96+
97+
- name: Run Deployment Script
98+
env:
99+
# Credentials and config for the script
100+
IBM_CLOUD_API_KEY: ${{ secrets.IBM_CLOUD_API_KEY }}
101+
IMAGE_URL: ${{ env.IBM_CLOUD_REGION }}.icr.io/${{ env.CR_NAMESPACE }}/${{ env.APP_NAME }}:${{ github.sha }}
102+
APP_NAME: ${{ env.APP_NAME }}
103+
104+
# Application environment variables
105+
SKIP_AUTH: ${{ secrets.SKIP_AUTH }}
106+
OIDC_DISCOVERY_ENDPOINT: ${{ secrets.OIDC_DISCOVERY_ENDPOINT }}
107+
IBM_CLIENT_ID: ${{ secrets.IBM_CLIENT_ID }}
108+
IBM_CLIENT_SECRET: ${{ secrets.IBM_CLIENT_SECRET }}
109+
FRONTEND_URL: ${{ secrets.FRONTEND_URL }}
110+
WATSONX_APIKEY: ${{ secrets.WATSONX_APIKEY }}
111+
WATSONX_INSTANCE_ID: ${{ secrets.WATSONX_INSTANCE_ID }}
112+
COLLECTIONDB_USER: ${{ secrets.COLLECTIONDB_USER }}
113+
COLLECTIONDB_PASS: ${{ secrets.COLLECTIONDB_PASS }}
114+
COLLECTIONDB_HOST: ${{ secrets.COLLECTIONDB_HOST }}
115+
COLLECTIONDB_PORT: ${{ secrets.COLLECTIONDB_PORT }}
116+
COLLECTIONDB_NAME: ${{ secrets.COLLECTIONDB_NAME }}
117+
VECTOR_DB: ${{ secrets.VECTOR_DB }}
118+
MILVUS_HOST: ${{ secrets.MILVUS_HOST }}
119+
MILVUS_PORT: ${{ secrets.MILVUS_PORT }}
120+
MILVUS_USER: ${{ secrets.MILVUS_USER }}
121+
MILVUS_PASSWORD: ${{ secrets.MILVUS_PASSWORD }}
122+
JWT_SECRET_KEY: ${{ secrets.JWT_SECRET_KEY }}
123+
LOG_LEVEL: "INFO"
124+
125+
run: |
126+
chmod +x ./scripts/deploy_codeengine.sh
127+
./scripts/deploy_codeengine.sh
128+
129+
smoke-test:
130+
needs: deploy-to-code-engine
131+
runs-on: ubuntu-latest
132+
steps:
133+
- name: Check out code
134+
uses: actions/checkout@v4
135+
136+
- name: Set up IBM Cloud CLI
137+
uses: ibm-cloud/sdk-action@v1
138+
139+
- name: Get application URL
140+
id: get-url
141+
run: |
142+
APP_URL=$(ibmcloud ce app get --name "${{ env.APP_NAME }}" --output json | jq -r '.status.latest_ready_revision_name' | head -1)
143+
if [ -n "$APP_URL" ]; then
144+
echo "url=$APP_URL" >> $GITHUB_OUTPUT
145+
else
146+
echo "Error: Could not get application URL" >&2
147+
exit 1
148+
fi
149+
150+
- name: Wait for application to be ready
151+
run: |
152+
echo "Waiting for application to be ready..."
153+
sleep 30
154+
155+
- name: Test health endpoint
156+
run: |
157+
APP_URL=$(ibmcloud ce app get --name "${{ env.APP_NAME }}" --output json | jq -r '.status.latest_ready_revision_name' | head -1)
158+
if [ -n "$APP_URL" ]; then
159+
echo "Testing health endpoint at: $APP_URL/health"
160+
if curl -f -s "$APP_URL/health" > /dev/null; then
161+
echo "✅ Health check passed"
162+
else
163+
echo "❌ Health check failed"
164+
exit 1
165+
fi
166+
else
167+
echo "❌ Could not determine application URL"
168+
exit 1
169+
fi
170+
171+
- name: Test API endpoint
172+
run: |
173+
APP_URL=$(ibmcloud ce app get --name "${{ env.APP_NAME }}" --output json | jq -r '.status.latest_ready_revision_name' | head -1)
174+
if [ -n "$APP_URL" ]; then
175+
echo "Testing API endpoint at: $APP_URL/api/v1/health"
176+
if curl -f -s "$APP_URL/api/v1/health" > /dev/null; then
177+
echo "✅ API health check passed"
178+
else
179+
echo "❌ API health check failed"
180+
exit 1
181+
fi
182+
else
183+
echo "❌ Could not determine application URL"
184+
exit 1
185+
fi

0 commit comments

Comments
 (0)