Skip to content

Run API tests once #306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 31, 2025
Merged

Run API tests once #306

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1fbdbff
WIP on running a single pass through service tests
abrookins Mar 31, 2025
3e0d79b
Merge branch '0.5.0' into run-one-api-test-suite
abrookins Mar 31, 2025
3acb3d3
fix workflow yml
abrookins Mar 31, 2025
373d1d2
Fix merge issue
abrookins Mar 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 9 additions & 25 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ env:
POETRY_VERSION: "1.8.3"

jobs:
prime-cache:
name: Prime HuggingFace Model Cache
service-tests:
name: Service Tests
runs-on: ubuntu-latest
env:
HF_HOME: ${{ github.workspace }}/hf_cache
Expand All @@ -38,10 +38,10 @@ jobs:
mkdir -p ~/.huggingface
echo '{"token":"${{ secrets.HF_TOKEN }}"}' > ~/.huggingface/token

- name: Set up Python 3.9
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11
cache: pip

- name: Install Poetry
Expand All @@ -58,7 +58,7 @@ jobs:
with:
credentials_json: ${{ secrets.GOOGLE_CREDENTIALS }}

- name: Run full test suite to prime cache
- name: Run full test suite and prime the HF cache
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
HF_HOME: ${{ github.workspace }}/hf_cache
Expand All @@ -80,13 +80,14 @@ jobs:
test:
name: Python ${{ matrix.python-version }} - ${{ matrix.connection }} [redis ${{ matrix.redis-version }}]
runs-on: ubuntu-latest
needs: prime-cache
needs: service-tests
env:
HF_HOME: ${{ github.workspace }}/hf_cache
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", 3.12, 3.13]
# 3.11 tests are run in the service-tests job
python-version: ["3.9", "3.10", 3.12, 3.13]
connection: ["hiredis", "plain"]
redis-version: ["6.2.6-v9", "latest", "8.0-M03"]

Expand Down Expand Up @@ -133,30 +134,14 @@ jobs:
with:
credentials_json: ${{ secrets.GOOGLE_CREDENTIALS }}

- name: Set HuggingFace token
run: |
mkdir -p ~/.huggingface
echo '{"token":"${{ secrets.HF_TOKEN }}"}' > ~/.huggingface/token

- name: Run tests
if: matrix.connection == 'plain' && matrix.redis-version == 'latest'
env:
HF_HOME: ${{ github.workspace }}/hf_cache
OPENAI_API_KEY: ${{ secrets.OPENAI_KEY }}
GCP_LOCATION: ${{ secrets.GCP_LOCATION }}
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
VOYAGE_API_KEY: ${{ secrets.VOYAGE_API_KEY }}
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_NAME }}
OPENAI_API_VERSION: ${{ secrets.OPENAI_API_VERSION }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
make test-all
make test

- name: Run tests (alternate)
if: matrix.connection != 'plain' || matrix.redis-version != 'latest'
Expand All @@ -182,7 +167,6 @@ jobs:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
docker run -d --name redis -p 6379:6379 redis/redis-stack-server:latest
make test-notebooks
if [[ "${{ matrix.python-version }}" > "3.9" ]]; then
make test-notebooks
else
Expand Down