:launch: Bascule vers quarto (#226)

* premeir test quarto * validate yaml * Continue avec quarto * modification badge vscode * passage sous forme de projet quarto * commence à tatonner sur action * hugo build * update * Ajoute des arguments au build * ajoute l'option par défaut wrap * modif * Ajoute une exception pour build * Workaround actions/checkout#760 * restructure page bundle * rename * retire bout problématique * Force rendering _index.qmd * mathjax * gère chemin images * ajoute image reshape * ajoute image join * Delete pandas_join.png * mv file * Ajoute le TP pandas * quarto * adapte le notebook * retire fix * file * correction * ajoute image * déplace deux TP * rename file * rename * typo folium * corrige typo dans chapitre geopandas * retour titre * output asis * corrige erreur * corrige erreur * dep * dep * surface * corrige des typos * carte arrondissements * figures by pandas * finalise cleaning gpd * arrange * Ajoute des ancres * essaie à nouveau la construction des notebooks * essaie à nouveau la construction des notebooks * change artifacts dir * change section name * ajoute corrections au worfjlow * ajoute corrections au worfjlow * Qmd maps * corrige clé * convert matplotlib tuto * clean qmd * echo false * feature engineering * ajoute index * Modélisation chapitre 2 * update * tp regression * ajoute feature selection * clustering * création fichier pipeline * finit le fichier * prb exec * corrige erreurs * import pandas * no more r * update * ajoute image * ajoute index * ajoute image scikit * typo * duplication image * duplication image * les deux premiers chapittes NLP * ajoute download * python LDA * error * Exos supp * chapitre elastic * chapitre elastic * rename utils * rename * refactorise * introgit * change anem * refactorise * refactorise * S3 * shell * typo * liste TP supprimée * update * update wordcloud * retour du wordcloud * erreur nom badge * update * raw shortcode * ajoute hack shortcode * update * en eval false * retire import brut image * tente un seul job pour les notebooks * nettoie tous les shortcodes * retire dossier notebooks * force * Automated changes * script to hack names * hack names * add move files * dir create * Automated changes * corrige typo * Automated changes * Automated changes * force commit * Automated changes * Automated changes * test ligne à ligne * Automated changes * Automated changes * trying differently * Automated changes * change git add * Automated changes * Automated changes * widlcard aussi là * updae * Automated changes * Automated changes * qmdise * tuto word2vec * les derniers TP * retire R * eval false pour tester * retour * eval false * eval false * retire * Automated changes * simplifie elastic * master * Automated changes * Automated changes * rewrite * Automated changes * Automated changes * clean * Automated changes * resub * Automated changes * Automated changes Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
linogaliana · May 25, 2022 · 12965ba · 12965ba
1 parent aa945cb
commit 12965ba
Show file tree

Hide file tree

Showing 54 changed files with 1,673 additions and 2,628 deletions.
diff --git a/.github/workflows/netlify-test.yaml b/.github/workflows/netlify-test.yaml
@@ -11,48 +11,40 @@ jobs:
   blogdown:
     name: Render-Blog
     runs-on: ubuntu-latest
-    container: linogaliana/python-datascientist:latest
+    container: linogaliana/python-datascientist-vstudio:quarto
     steps:
       - uses: actions/checkout@v2
         with:
           submodules: true
           fetch-depth: 0
           ref: ${{ github.event.pull_request.head.ref }}
           repository: ${{github.event.pull_request.head.repo.full_name}}
-      - name: Install rmarkdown
-        env:
-          GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
-        run: |
-          Rscript -e 'remotes::install_github("yihui/xfun")'
-          Rscript -e 'remotes::install_github("yihui/knitr")'
-          Rscript -e 'install.packages(c("rmarkdown", "here"))'
-      - name: install hugo
-        run: Rscript -e 'blogdown::install_hugo("0.83.0", force = TRUE)'
-      - name: Install Python
-        run: |
-          Rscript -e 'print(blogdown:::hugo_version())'
-          Rscript -e "install.packages(c('remotes', 'reticulate'))"
-          Rscript -e "install.packages(c('here'))"
-      - shell: bash -l {0}
+      - name: Configure safe.directory  # Workaround for actions/checkout#760
+        run: git config --global --add safe.directory /__w/python-datascientist/python-datascientist
+      - shell: bash
         run: |
           conda info
           conda list
       - name: Build to md
         run: |
-          git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
-          cat diff.txt
-          Rscript -e 'source("./build/build_light.R")'
+          quarto render --to hugo
+          mv content/course/manipulation/index.md content/course/manipulation/_index.md
+          mv content/course/visualisation/index.md content/course/visualisation/_index.md
+          mv content/course/modelisation/index.md content/course/modelisation/_index.md
+          #git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
+          #cat diff.txt
+          #Rscript -e 'source("./build/build_light.R")'
+          hugo mod graph
+          hugo -D --themesDir themes -t github.com
       - name: Clean files with Python function
         run: |
-         rm -rf "./temp"
-         mkdir -p temp
-         mkdir -p notebooks
          pip install matplotlib wordcloud nltk
-         python build/cleanmd.py
          python build/wc_website.py
-      - name: Render blog
+         python build/post-build.py
+      - name: Copy notebooks in dedicated directory
         run: |
-          Rscript -e 'source("./build/netlify.R")'
+         mkdir -p notebooks
+         python build/move_files.py notebooks
       - name: Install npm
         if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
         uses: actions/setup-node@v2          
@@ -74,28 +66,10 @@ jobs:
         with:
           name: Website
           path: public/
-      - name: Convert in ipynb with Jupytext
-        run: |
-         pip install jupytext
-         cd ./temp
-         for i in $(find . -type f \( -iname "*.Rmd" \)); do \
-            j="${i%.Rmd}.md" ;
-            echo "$j" ;
-            if [ -e $j ]
-            then
-              echo "Converting $j"; \
-              #jupytext --to py --execute "$i"
-              jupytext --to ipynb "$j" ;
-              k="${j%.md}.ipynb" ;
-            fi            
-         done
-         mkdir -p ../notebooks
-         cp -R . ../notebooks
-         cd ..
       - uses: actions/upload-artifact@v2
         with:
           name: Notebooks
-          path: temp/
+          path: notebooks/
       - name: Commit new notebooks
         if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
         env:
@@ -106,67 +80,32 @@ jobs:
          # git config user.email github-actions@github.com
          #git checkout ${BRANCHE_REF}
          git status
-         git add notebooks/**/*.ipynb;
+         find notebooks -type f -name "*.ipynb" | xargs git add
          git commit -m "Automated changes";
          git pull
          git push
   corrections:
+    name: Render-Blog
     runs-on: ubuntu-latest
-    container: linogaliana/python-datascientist:latest
+    container: linogaliana/python-datascientist-vstudio:quarto
     steps:
       - uses: actions/checkout@v2
         with:
           submodules: true
           fetch-depth: 0
           ref: ${{ github.event.pull_request.head.ref }}
           repository: ${{github.event.pull_request.head.repo.full_name}}
-      - name: Install rmarkdown
-        env:
-          GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
-        run: |
-          Rscript -e 'remotes::install_github("yihui/xfun")'
-          Rscript -e 'remotes::install_github("yihui/knitr")'
-          Rscript -e 'install.packages(c("rmarkdown"))'
-      - name: Install Python
-        run: |
-          Rscript -e "install.packages(c('remotes', 'reticulate'))"
-          Rscript -e "install.packages(c('here'))"
-      - shell: bash -l {0}
+      - name: Configure safe.directory  # Workaround for actions/checkout#760
+        run: git config --global --add safe.directory /__w/python-datascientist/python-datascientist
+      - shell: bash
         run: |
           conda info
           conda list
-      - name: Modif Rmd echo status
-        run: |
-         python build/modifmd.py
-      - name: Build to md
-        run: |
-          git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
-          cat diff.txt
-          Rscript -e 'source("./build/build_light.R")'
-      - name: Clean files with Python function
-        run: |
-         rm -rf "./temp"
-         mkdir -p temp
-         mkdir -p corrections
-         python build/cleanmd.py
       - name: Convert in ipynb with Jupytext
         run: |
-         pip install jupytext
-         cd ./temp
-         for i in $(find . -type f \( -iname "*.Rmd" \)); do \
-            j="${i%.Rmd}.md" ;
-            echo "$j" ;
-            if [ -e $j ]
-            then
-              echo "Converting $j"; \
-              #jupytext --to py --execute "$i"
-              jupytext --to ipynb "$j" ;
-              k="${j%.md}.ipynb" ;
-            fi            
-         done
-         mkdir -p ../corrections
-         cp -R . ../corrections
-         cd ..
+         quarto render content --to ipynb --execute
+         mkdir -p corrections
+         python build/move_files.py corrections
       - uses: actions/upload-artifact@v2
         with:
           name: Corrections
@@ -181,8 +120,87 @@ jobs:
          # git config user.email github-actions@github.com
          #git checkout ${BRANCHE_REF}
          git status
-         git add corrections/**/*.ipynb;
+         find corrections -type f -name "*.ipynb" | xargs git add
          git commit -m "Automated changes";
          git pull
          git push
 
+
+
+
+#  corrections:
+#    runs-on: ubuntu-latest
+#    container: linogaliana/python-datascientist:latest
+#    steps:
+#      - uses: actions/checkout@v2
+#        with:
+#          submodules: true
+#          fetch-depth: 0
+#          ref: ${{ github.event.pull_request.head.ref }}
+#          repository: ${{github.event.pull_request.head.repo.full_name}}
+#      - name: Install rmarkdown
+#        env:
+#          GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
+#        run: |
+#          Rscript -e 'remotes::install_github("yihui/xfun")'
+#          Rscript -e 'remotes::install_github("yihui/knitr")'
+#          Rscript -e 'install.packages(c("rmarkdown"))'
+#      - name: Install Python
+#        run: |
+#          Rscript -e "install.packages(c('remotes', 'reticulate'))"
+#          Rscript -e "install.packages(c('here'))"
+#      - shell: bash -l {0}
+#        run: |
+#          conda info
+#          conda list
+#      - name: Modif Rmd echo status
+#        run: |
+#         python build/modifmd.py
+#      - name: Build to md
+#        run: |
+#          git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
+#          cat diff.txt
+#          Rscript -e 'source("./build/build_light.R")'
+#      - name: Clean files with Python function
+#        run: |
+#         rm -rf "./temp"
+#         mkdir -p temp
+#         mkdir -p corrections
+#         python build/cleanmd.py
+#      - name: Convert in ipynb with Jupytext
+#        run: |
+#         pip install jupytext
+#         cd ./temp
+#         for i in $(find . -type f \( -iname "*.Rmd" \)); do \
+#            j="${i%.Rmd}.md" ;
+#            echo "$j" ;
+#            if [ -e $j ]
+#            then
+#              echo "Converting $j"; \
+#              #jupytext --to py --execute "$i"
+#              jupytext --to ipynb "$j" ;
+#              k="${j%.md}.ipynb" ;
+#            fi            
+#         done
+#         mkdir -p ../corrections
+#         cp -R . ../corrections
+#         cd ..
+#      - uses: actions/upload-artifact@v2
+#        with:
+#          name: Corrections
+#          path: corrections/
+#      - name: Commit new notebooks
+#        if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
+#        env:
+#          BRANCHE_REF: ${{ github.event.pull_request.head.ref }}
+#        run: |
+#         git config user.name 'github-actions[bot]'
+#         git config user.email 'github-actions[bot]@users.noreply.github.com'
+#         # git config user.email github-actions@github.com
+#         #git checkout ${BRANCHE_REF}
+#         git status
+#         git add corrections/**/*.ipynb;
+#         git commit -m "Automated changes";
+#         git pull
+#         git push
+
diff --git a/.github/workflows/prod.yml b/.github/workflows/prod.yml
@@ -10,51 +10,47 @@ jobs:
   blogdown:
     name: Render-Blog
     runs-on: ubuntu-latest
-    container: linogaliana/python-datascientist:latest
-    if: "!contains(github.event.commits[0].message, '[skip ci]')"
+    container: linogaliana/python-datascientist-vstudio:quarto
     steps:
-      - uses: actions/checkout@v1
+      - uses: actions/checkout@v2
         with:
           submodules: true
-      - name: Install rmarkdown
-        env:
-          GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
-        run: |
-          Rscript -e 'remotes::install_github("yihui/xfun")'
-          Rscript -e 'remotes::install_github("yihui/knitr")'
-          Rscript -e 'install.packages(c("rmarkdown", "here"))'
-      - name: install hugo
-        run: Rscript -e 'blogdown::install_hugo("0.83.0", force = TRUE)'
-      - name: Install Python
-        run: |
-          Rscript -e 'print(blogdown:::hugo_version())'
-          Rscript -e "install.packages(c('remotes', 'reticulate'))"
-          Rscript -e "install.packages(c('here'))"
-      - shell: bash -l {0}
+          fetch-depth: 0
+          ref: ${{ github.event.pull_request.head.ref }}
+          repository: ${{github.event.pull_request.head.repo.full_name}}
+      - name: Configure safe.directory  # Workaround for actions/checkout#760
+        run: git config --global --add safe.directory /__w/python-datascientist/python-datascientist
+      - shell: bash
         run: |
           conda info
           conda list
-      - name: Render blog
+      - name: Build to md
         run: |
-          Rscript -e 'source("./build/build.R")'
-      - name: Create
+          quarto render --to hugo
+          mv content/course/manipulation/index.md content/course/manipulation/_index.md
+          mv content/course/visualisation/index.md content/course/visualisation/_index.md
+          mv content/course/modelisation/index.md content/course/modelisation/_index.md
+          #git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
+          #cat diff.txt
+          #Rscript -e 'source("./build/build_light.R")'
+          hugo mod graph
+          hugo -D --themesDir themes -t github.com
+      - name: Clean files with Python function
         run: |
-         rm -rf "./temp"
-         mkdir -p temp
-         mkdir -p notebooks
          pip install matplotlib wordcloud nltk
-         python build/cleanmd.py
          python build/wc_website.py
-      - name: Render blog
+         python build/post-build.py
+      - name: Copy notebooks in dedicated directory
         run: |
-          Rscript -e 'source("./build/netlify.R")'
+         mkdir -p notebooks
+         python build/move_files.py notebooks
       - name: Install npm
-        if: ${{ github.repository == 'linogaliana/python-datascientist' }}
+        if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
         uses: actions/setup-node@v2          
         with:
           node-version: '12'        
       - name: Deploy to Netlify
-        if: ${{ github.repository == 'linogaliana/python-datascientist' }}
+        if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
         # NETLIFY_AUTH_TOKEN and NETLIFY_SITE_ID added in the repo's secrets
         env:
           NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
@@ -63,6 +59,7 @@ jobs:
         run: |
           npm init -y
           npm install --unsafe-perm=true netlify-cli -g
+          netlify init
           netlify deploy --prod --dir="public" --message "Deploy master"
       - uses: actions/upload-artifact@v1
         with:

diff --git a/.gitignore b/.gitignore
@@ -141,3 +141,5 @@ pip-selfcheck.json
 # End of https://www.toptal.com/developers/gitignore/api/pycharm,venv
 .Rproj.user
 .Rhistory
+
+/.quarto/
diff --git a/_quarto.yml b/_quarto.yml
@@ -0,0 +1,21 @@
+project:
+  title: "python-datascientist"
+  pre-render: utils.py
+  render:
+    - content/**/*.qmd
+
+validate-yaml: false
+jupyter: python3
+
+wrap: preserve
+keep-ipynb: true
+cache: true
+
+format:
+  hugo:
+     html-math-method: 
+      method: mathjax
+  ipynb: default
+
+#bibliography: references.bib
+