Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make settings.py optional #3163

Merged

Conversation

miguel-ortiz-marin
Copy link
Contributor

@miguel-ortiz-marin miguel-ortiz-marin commented Oct 11, 2023

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

  • Hacktoberfest PR 👻 !
  • Currently settings.py must be provided for a Kedro Project. kedro new always ship a settings.py with commented default. We should update the functionality so that when a user deleted settings.py, kedro run still works.
  • Closes Make settings.py optional #3050

Development notes

  • Changed test_settings.py and framework/project/__init__.py
  • The core Idea is to check if the file exists or not with importlib.util.find_spec before trying to validate it .
  • I ran previous test_settings and included a new test and mock mock_package_name_without_settings_file and test_validate_settings_without_settings_file.
  • I installed a local version of the repo with pip install -e . and then ran a kedro new --starter=spaceflights. Then I validated that kedro run worked with, and without a settings.py file.
  • mypy doesn't find importlib.util so I added a # type: ignore in respective lines

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

  • Read the contributing guidelines
  • Signed off each commit with a Developer Certificate of Origin (DCO)
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes
  • Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

@ankatiyar
Copy link
Contributor

@miguel-ortiz-marin Thanks for working on this, I know you haven't opened it up for review yet but make sure your commits are all signed off! Follow the instructions here - https://github.com/kedro-org/kedro/pull/3163/checks?check_run_id=17608910410

Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>
Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>
@miguel-ortiz-marin miguel-ortiz-marin force-pushed the feature/optional_setting_py branch from 5726e7b to b048ee8 Compare October 11, 2023 18:08
@miguel-ortiz-marin
Copy link
Contributor Author

miguel-ortiz-marin commented Oct 11, 2023

Hi @ankatiyar nice catch! I had written git commit -S instead of -s haha.

I now pushed new commits that are passing(link to ci) the unit tests but the CI is failing nonetheless after updating the branch from main ⁉️ (I think we have to wait for main to be updated again)

I'm going to mark it ready for review! I'll stay tuned for the comments

Thank you! 👻

@miguel-ortiz-marin miguel-ortiz-marin marked this pull request as ready for review October 11, 2023 18:35
@ankatiyar
Copy link
Contributor

Did a quick manual test and things seem to be working fine and all the tests are passing 🙌🏾 but the code coverage is not complete. Could you look into this @miguel-ortiz-marin? https://github.com/kedro-org/kedro/actions/runs/6486331734/job/17633747121?pr=3163

@ankatiyar ankatiyar added the Community Issue/PR opened by the open-source community label Oct 12, 2023
Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>
…-marin/kedro into feature/optional_setting_py
@miguel-ortiz-marin
Copy link
Contributor Author

@ankatiyar Awesome! I was missing one line in a unit test.

It looks now like the test pass in linux but build is failing in windows 🤔

Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this contribution @miguel-ortiz-marin ! I left one minor comment and also wanted to ask if you can add this change to the release notes. It should go under the next non-breaking release 0.18.14.

You can ignore the breaking tests on CircleCI. As long as all the tests on Github Actions run everything is fine 😄

importlib.import_module(f"{PACKAGE_NAME}.settings")
else:
logger = logging.getLogger(__name__)
logger.warning("No settings.py found, defaults will be used.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestion to get nicer logs highlighting:

Suggested change
logger.warning("No settings.py found, defaults will be used.")
logger.warning("No 'settings.py' found, defaults will be used.")

Copy link
Contributor

@AhdraMeraliQB AhdraMeraliQB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up! ⭐

miguel-ortiz-marin and others added 2 commits October 13, 2023 07:34
Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>
@astrojuanlu astrojuanlu enabled auto-merge (squash) October 13, 2023 13:45
@astrojuanlu astrojuanlu disabled auto-merge October 13, 2023 13:45
@miguel-ortiz-marin
Copy link
Contributor Author

miguel-ortiz-marin commented Oct 13, 2023

Looks like all's set now @astrojuanlu ! Thank you! 🚀

@astrojuanlu astrojuanlu merged commit e1823ed into kedro-org:main Oct 13, 2023
40 of 41 checks passed
@astrojuanlu
Copy link
Member

Congrats @miguel-ortiz-marin ! 🎉

@miguel-ortiz-marin
Copy link
Contributor Author

Thankss! This was my first open source contribution !! To many more 🏎️ !

adamkells pushed a commit to adamkells/kedro that referenced this pull request Oct 30, 2023
* Make settings.py optional

Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>

* Fix unit tests

Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>

* Fix coverage

Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>

* Update RELEASE.md and logging

Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>

---------

Signed-off-by: Miguel Ortiz <miguel_ortiz_marin@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Adam Kells <adamjkells93@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Community Issue/PR opened by the open-source community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make settings.py optional
5 participants