Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add python CI #3

Closed
wants to merge 12 commits into from
53 changes: 53 additions & 0 deletions .github/workflows/python-package-conda.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# This is copied from the python starter kit for github actions:
# https://github.com/actions/starter-workflows/blob/main/ci/python-package.yml
name: Azure End to End Testing with existing Jars

# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

on:
pull_request:
types:
- opened
- reopened
- synchronize
- ready_for_review

jobs:
build:

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9"]

steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Install Feathr Package
run: |
python -m pip install -e ./feathr_project/
- name: Test with pytest
env:
REDIS_PASSWORD: ${{secrets.REDIS_PASSWORD}}
AZURE_CLIENT_ID: ${{secrets.AZURE_CLIENT_ID}}
AZURE_TENANT_ID: ${{secrets.AZURE_TENANT_ID}}
AZURE_CLIENT_SECRET: ${{secrets.AZURE_CLIENT_SECRET}}
run: |
cd feathr_project
pytest
4 changes: 2 additions & 2 deletions feathr_project/test/test_azure_e2e.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ def test_feathr_online_store():
job_res = client.materialize_features()
# just assume the job is successful without validating the actual result in Redis. Might need to consolidate
# this part with the test_feathr_online_store test case
client.wait_job_to_finish(timeout_sec=300)
client.wait_job_to_finish(timeout_sec=600)
res = client.online_get_features('nycTaxiDemoFeature', '265', ['f_location_avg_fare', 'f_location_max_fare'])
# just assme there are values. We don't hard code the values for now for testing
# the correctness of the feature generation should be garunteed by feathr runtime.
Expand Down Expand Up @@ -55,7 +55,7 @@ def test_feathr_get_historical_features():
client = FeathrClient()

returned_spark_job = client.join_offline_features()
res_url = client.get_job_result_uri(block=True, timeout_sec=500)
res_url = client.get_job_result_uri(block=True, timeout_sec=600)
tmp_dir = tempfile.TemporaryDirectory()
client.feathr_synapse_laucher.download_result(result_path=res_url, local_folder=tmp_dir.name)
dataframe_list = []
Expand Down