Introduce proper formatting on python ,bash and yaml files (kubeflow#…

…2774)

* Add balck github action to format python files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Formatted python files from the balck formatter

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add github action to format yaml files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add step YAML Formatting Guidelines to yaml_formatter.yaml file

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* made chnages to run next steps although the previous step fail

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* switch steps

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* added shellCheck for bash formatting

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Changed code only to lint yaml files inside the common and example folder

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Changed main to master

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Run yamllint on files which is chnaged in PR only

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Changed 'Proper formatting on python files' github workflow only to run for python files in common and example folder

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add shellcheckrc file

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Disable SC1017,SC2086,SC2070 rules when shellcheck and did changes in sh files to address the formattings according to shellcheck

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* made chnages in sh files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* made chnages in sh files to address formattings

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* made chnages in sh files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* disable SC2046 rule

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* disable SC2006 and SC2155 rule

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Change shellcheck only to run for PR chnaged files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Address a issue with bash_formatter.yaml

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Changed origin/main to origin/master

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Formatted python files from the balck formatter

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add github action to format yaml files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add step YAML Formatting Guidelines to yaml_formatter.yaml file

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* made chnages to run next steps although the previous step fail

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* switch steps

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add shellcheckrc file

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Disable SC1017,SC2086,SC2070 rules when shellcheck and did changes in sh files to address the formattings according to shellcheck

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* made chnages in sh files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Change shellcheck only to run for PR chnaged files

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Ensure the full history is fetched

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Ensure the full history is fetched in bash_formatter

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* fix SC2148 (error)

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* remove commented lines in bash_formatter.yaml

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* remove formatting chnages done to app folder content

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add guidlines to how to format python files according to black formatter

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

* Add guidlines to how to format bash files according to shellcheck formatter

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

---------

Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com>

.github/workflows/bash_formatter.yaml

@@ -0,0 +1,40 @@
+name: Proper Formatting on bash files
+
+on: [push, pull_request]
+
+jobs:
+  format_bash_files:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        fetch-depth: 0  
+
+    - name: Install ShellCheck
+      run: sudo apt install -y shellcheck
+
+    - name: Bash Formatting Guidelines
+      run: |
+        echo "### Bash Files Formatting Guidelines ###
+        If there are errors and warnings regarding your bash files,
+        You can check that error code definitions in https://www.shellcheck.net/wiki/ site.
+        You can correct them using the https://www.shellcheck.net/ site. 
+        You have to ignore disable errors in .shellcheckrc file.
+        "
+
+    - name: Fetch master branch
+      run: git fetch origin master
+
+    - name: Set up changed files
+      run: |
+        git diff --name-only origin/master...HEAD | grep -E '^.*\.sh$' | grep -v '^apps/' > changed_files_in_PR.txt || true
+    
+    - name: Display changed files
+      run: cat changed_files_in_PR.txt
+
+    - name: Run ShellCheck on changed files
+      run: |
+        cat changed_files_in_PR.txt | xargs -I {} shellcheck {}
+      shell: bash
+
+

.github/workflows/python_formatter.yaml

@@ -0,0 +1,32 @@
+name: Proper Formatting on Python files
+
+on: [push, pull_request]
+
+jobs:
+  format_python_files:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Python Files Formatting Guidelines
+        run: |
+          echo "### Python Files Formatting Guidelines ###
+          If there is a formatting errors in your python files, 
+          1. First install black 
+             It requires Python 3.8+ to run.
+             Install with  "pip install black" and if you use pipx, install Black with "pipx install black"
+             If you want to format Jupyter Notebooks, install with pip install "black[jupyter]"
+
+          2. Run the command 
+              "python -m black {source_file_or_directory}" or
+              "black {source_file_or_directory}" 
+             to format python files. 
+          "
+      
+      - uses: psf/black@stable
+        with:
+          src: |
+            ./common
+            ./example
+
+    

.github/workflows/yaml_formatter.yaml

@@ -0,0 +1,59 @@
+name: Proper Formatting on YAML files
+
+on: [push, pull_request]
+
+jobs:
+  format_YAML_files:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        fetch-depth: 0 
+
+    - name: Install yamllint
+      run: pip install yamllint
+
+    - name: YAML Formatting Guidelines
+      run: |
+        echo "### YAML Formatting Guidelines ###
+        If there is a formatting error in your YAML file, you will see errors like the one below:
+        'Error: 6:4 [indentation] wrong indentation: expected 2 but found 3'
+        
+        To fix these errors, refer to the YAML formatting rules at:
+        https://yamllint.readthedocs.io/en/stable/rules.html#
+        
+        Search for the keyword inside the brackets [] in the error message. In this example, it's 'indentation'.
+        
+        Note: Some rules have been customized in the '.yamllint.yaml' file. Below is the content of that file:
+
+        extends: default
+
+        rules:
+          document-start:
+            present: false
+          document-end:
+            present: false
+          indentation:
+            indent-sequences: false
+          line-length:
+            max: 400
+        "
+
+    - name: Fetch master branch
+      run: git fetch origin master
+
+    - name: Set up changed files
+      run: |
+        git diff --name-only origin/master...HEAD | grep -E '^common/.*\.ya?ml$|^example/.*\.ya?ml$' > changed_files_in_PR.txt || true
+    
+    - name: Display changed files
+      run: cat changed_files_in_PR.txt
+
+    - name: Run yamllint on changed files
+      run: |
+        chmod +x ./run_yamllint.sh
+        ./run_yamllint.sh
+      shell: bash
+
+
+

.shellcheckrc

@@ -0,0 +1,2 @@
+# ~/.shellcheckrc
+disable=SC1017,SC2086,SC2070,SC2046,SC2155,SC2006

.yamllint.yaml

@@ -0,0 +1,12 @@
+# It extends the default conf by adjusting some options.
+extends: default
+
+rules:
+  document-start:
+    present: false
+  document-end:
+    present: false
+  indentation:
+    indent-sequences: false
+  line-length:
+    max: 400

contrib/kserve/tests/test_sklearn.py

@@ -50,7 +50,9 @@ def test_sklearn_kserve():
         spec=V1beta1InferenceServiceSpec(predictor=predictor),
     )
 
-    kserve_client = KServeClient(config_file=os.environ.get("KUBECONFIG", "~/.kube/config"))
+    kserve_client = KServeClient(
+        config_file=os.environ.get("KUBECONFIG", "~/.kube/config")
+    )
     kserve_client.create(isvc)
     kserve_client.wait_isvc_ready(service_name, namespace=KSERVE_TEST_NAMESPACE)
     res = predict(service_name, "./data/iris_input.json")

hack/synchronize-istio-cni-manifests.sh

@@ -1,5 +1,5 @@
-@ -1,88 +0,0 @@
 #!/usr/bin/env bash
+@ -1,88 +0,0 @@
 
 # This script aims at helping create a PR to update the manifests of the
 # knative.

hack/synchronize-istio-manifests.sh

@@ -1,4 +1,5 @@
-# #!/usr/bin/env bash
+#!/usr/bin/env bash
+
 
 # # This script aims at helping create a PR to update the manifests of Istio
 # # This script:

hack/synchronize-kserve-web-app-manifests.sh

@@ -24,7 +24,7 @@ if [ -n "$(git status --porcelain)" ]; then
   echo "WARNING: You have uncommitted changes"
 fi
 
-if [ `git branch --list $BRANCH` ]
+if [ "$(git branch --list $BRANCH)" ]
 then
    echo "WARNING: Branch $BRANCH already exists."
 fi
@@ -39,11 +39,11 @@ echo "Checking out in $SRC_DIR to $COMMIT..."
 
 # Checkout the Model Registry repository
 mkdir -p $SRC_DIR
-cd $SRC_DIR
+cd $SRC_DIR || exit
 if [ ! -d "models-web-app/.git" ]; then
     git clone https://github.com/kserve/models-web-app.git
 fi
-cd $SRC_DIR/models-web-app
+cd $SRC_DIR/models-web-app || exit
 if ! git rev-parse --verify --quiet $COMMIT; then
     git checkout -b $COMMIT
 else
@@ -71,7 +71,7 @@ DST_TXT="\[$COMMIT\](https://github.com/kserve/models-web-app/tree/$COMMIT/confi
 sed -i "s|$SRC_TXT|$DST_TXT|g" "${MANIFESTS_DIR}"/README.md
 
 echo "Committing the changes..."
-cd $MANIFESTS_DIR
+cd $MANIFESTS_DIR || exit
 git add contrib/kserve/models-web-app
 git add README.md
 git commit -s -m "Update kserve models web application manifests from ${COMMIT}"