[AUTO-MERGE] Merge main into develop via merge-main-to-develop #2654
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
…onal targets (#2557) Ready at last. Still some conversation to be had about how we reflect "confidence" in the plugins we describe (what version they were last tested against) but the text is ready and better than what's there currently, so let's go!
#2568) * Stop producing `.egg` files from `kedro package` Fix gh-2273. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Upgrade import-linter Fix gh-2570. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Partially remove reliance on setup.py for micropackaging Incomplete effort, see gh-2414. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Revert "Partially remove reliance on setup.py for micropackaging" This reverts commit fd3efa8. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
…ntime_params properly (#2467) * Fix typehint Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * test push Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * POC of fix to solve the runtime param resolution problem Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix OmegaConfigLoadaer - resolve runtime_params early Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Delegate the intialization of runtime_params to AbstractConfigLoader Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add test for interpolated value and globals Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * add more test and linting Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * refactor Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * update release note Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Apply comments and refactor the test Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update RELEASE.md Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com> --------- Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>
* Partially revert #2378 as OCL no longer returning DictConfig Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Revert #2176 with special logging handling Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * remove unused import Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
…ibution section (#2579) * modify databricks testing guide for contributors Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/contribution/development_for_databricks.md Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com> * Update development_for_databricks.md --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
… file is empty (#2584) * Allow empty config files Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Remove condition for empty config Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Revert "Remove condition for empty config" This reverts commit 7961155. * Update release notes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add unit test Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
* Proposal of updated settings.py Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Update settings after comment Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
* Add metadata attribute to kedro.io datasets Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
* Update pipeline hooks docstrings Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com> * Acknowledge community contribution --------- Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Add debug level logging for hook execution order Fix gh-1942. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Clarify role of setuptools in hooks Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Remove mentions to outdated and unused `pkg_resources` Fix gh-2391. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Adapt example hook to spec signature Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Grammar and style fixes in docs Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Revert "Adapt example hook to spec signature" This reverts commit ac1b495. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
…d" with "legacy" (#2606)
* FIX: Typo in markdown for kedro notebooks docs Signed-off-by: debugger24 <rahulcomp24@gmail.com> * Acknowledge community contribution --------- Signed-off-by: debugger24 <rahulcomp24@gmail.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Add the Kedro RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Clean up imports and comments Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Map arguments to add flexibility to customize arguments available in RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Clean up rich_logger.py and unused imports Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add docs - first iteration Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix imports and small refactoring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Improve the doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update the project's template Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * format docstring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Apply Jo's comment on the doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix the link with RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Apply suggestions from code review Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com> * Fix broken doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * update template defaults Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * refactoring RichHandler and improve doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix broken test due to new default RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Move kedro.extras.logging.RichHandler -> kedro.logging.RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * add kedro/logging.py and remove the unnecessary import that cause namespace collision Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Refactor test Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add more tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix lots of import, docstring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Reorder docstring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Link options to doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Clean up doc and unused logger Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix spelling `customise` Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Refactor test with fixture Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Remove unnecessary copy() because it is now a fixture Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Split up test_rich_traceback_configuration() to smaller tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add RELEASE.md Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix template logging
* Mention awesome-kedro in docs See kedro-org/kedro-devrel#40 * Update CONTRIBUTING.md * Spelling and style fixes Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
* Update SnowparkTableDataSet docs * Add missing snowpark dependencies for docs building Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Update __init__.py * Update release note Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Update Version Numbers across docs Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Update RELEASE.md * Update Release Note Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
…#2595) * Add deployment workflow page Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add table of contents, entry point guide, data and conf upload guide Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add detailed instructions for creating a job on Databricks Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add images and automated deployment resources Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Remove use of 'allows', add summary Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Remove link to missing image Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add deployment workflow to toctree Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Lint and fix missing link Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Minor style, syntax and grammar improvements Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Fixes for correctness during validation Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add instructions for creating log output location Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Lint Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Lint databricks_run Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Minor wording change in reference to logs Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Modify reference to Pyspark-Iris Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Fix linter errors to enable docs build for inspection Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update build-docs.sh * Fix broken link Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Remove spurious word Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add advantages subheading Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add alternative ways to upload data to DBFS Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Move note on unpackaged config and data Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Fix broken links Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Move databricks back into deployment section Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Remove references to PySpark Iris (pyspark-iris) starter Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Graphics links fixes, revise titles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Fix broken internal link Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Fix links broken by new folder Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Remove logs directory Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update image of final job configuration Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add full stops in list. Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Fix conda environment name. Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Modify wording and image for creating a new job cluster Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Modify wording in guide to create new job cluster Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Remove --upgrade option Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Add both ways of creating a new job Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> --------- Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Update docs font family to Inter Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Match weights to previous Google font Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update logo images; revert css formatting for easier PR review Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update text styles Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Add trailing whitespace (?) Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update Kedro image path Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update font size Signed-off-by: Tynan DeBold <thdebold@gmail.com> * update font to font-family: 'Inter', sans-serif; Signed-off-by: huongg <huongg1409@gmail.com> * testing with important for font-family Signed-off-by: huongg <huongg1409@gmail.com> * Make project name lowercase in docs Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * revert back to kedro/develop Signed-off-by: huongg <huongg1409@gmail.com> --------- Signed-off-by: Tynan DeBold <thdebold@gmail.com> Signed-off-by: huongg <huongg1409@gmail.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com> Co-authored-by: huongg <huongg1409@gmail.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
#2621) * Enable variable interpolation in catalog by escaping _ Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
* add verification file and change to conf Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Fix conf.py so google file is moved to root of docs build Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
* Update docs font family to Inter Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Match weights to previous Google font Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update logo images; revert css formatting for easier PR review Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update text styles Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Add trailing whitespace (?) Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update Kedro image path Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update font size Signed-off-by: Tynan DeBold <thdebold@gmail.com> * update font to font-family: 'Inter', sans-serif; Signed-off-by: huongg <huongg1409@gmail.com> * testing with important for font-family Signed-off-by: huongg <huongg1409@gmail.com> * update assets for experiment tracking, docs, pipeline, focus mode, matplotlib and plotly * update modular pipelines section Signed-off-by: huongg <huongg1409@gmail.com> * autoreload Signed-off-by: huongg <huongg1409@gmail.com> * reupload autoreload gif Signed-off-by: huongg <huongg1409@gmail.com> * revert back to develop branch Signed-off-by: huongg <huongg1409@gmail.com> * gif for ET comparison Signed-off-by: huongg <huongg1409@gmail.com> * remove jupyter notebook screenshot since its not used anywhere Signed-off-by: huongg <huongg1409@gmail.com> * Update databricks viz demo Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * pipeline_show_metrics gif Signed-off-by: huongg <huongg1409@gmail.com> * update different gif format Signed-off-by: huongg <huongg1409@gmail.com> * point to main instead of develop for kedro-banner Signed-off-by: huongg <huongg1409@gmail.com> --------- Signed-off-by: Tynan DeBold <thdebold@gmail.com> Signed-off-by: huongg <huongg1409@gmail.com> Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Co-authored-by: Tynan DeBold <thdebold@gmail.com> Co-authored-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Jannic Holzer <jannic.holzer@quantumblack.com>
…2647) Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
merelcht
requested review from
noklam
and removed request for
astrojuanlu,
stichbury and
yetudada
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
…kedro into merge-main-to-develop Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
noklam
approved these changes
Jun 7, 2023
5 tasks
merelcht
approved these changes
Jun 7, 2023
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A new change in main cannot be merged into develop as part of the regular sync job, hence this PR. Please resolve the conflicts manually, and make sure to obtain 2 approvals once the builds pass.
IMPORTANT NOTICE
Please let CircleCI merge this PR automatically, with merge commit enabled.