Merge template pyproject.toml into one (#2926)

* Turn default template into src-layout Python package

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix tests after metadata and tests moved

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove leftover mention of environment.yml

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Move src/requirements.txt one level up

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Make `kedro package` backwards compatible

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add template changes to release notes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add test for coverage

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* lint

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Fix outdated doc link

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Co-authored-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

RELEASE.md

@@ -21,6 +21,8 @@
 * Removed deprecated `kedro.extras.ColorHandler`.
 * The Kedro IPython extension is no longer available as `%load_ext kedro.extras.extensions.ipython`; use `%load_ext kedro.ipython` instead.
 * Anonymous nodes are given default names of the form `<function_name>([in1;in2;...]) -> [out1;out2;...]`, with the names of inputs and outputs separated by semicolons.
+* The default project template now has one `pyproject.toml` at the root of the project (containing both the packaging metadata and the Kedro build config).
+* The `requirements.txt` in the default project template moved to the root of the project as well (hence dependencies are now installed with `pip install -r requirements.txt` instead of `pip install -r src/requirements.txt`).
 
 ## Migration guide from Kedro 0.18.* to 0.19.*
 ### DataSets

docs/source/deployment/airflow_astronomer.md

@@ -84,7 +84,7 @@ To follow this tutorial, ensure you have the following:
     pip install kedro-airflow~=0.4
     ```
 
-5. Run `pip install -r src/requirements.txt` to install all dependencies.
+5. Run `pip install -r requirements.txt` to install all dependencies.
 
 ### Deployment process
 

docs/source/deployment/aws_step_functions.md

@@ -55,7 +55,7 @@ The rest of the tutorial will explain each step in the deployment process above
 
 * Create a `conf/aws` directory in your Kedro project
 * Put a `catalog.yml` file in this directory with the following content
-* Ensure that you have `s3fs>=0.3.0,<0.5` defined in your `src/requirements.txt` so the data can be read from S3.
+* Ensure that you have `s3fs>=0.3.0,<0.5` defined in your `requirements.txt` so the data can be read from S3.
 
 <details>
 <summary><b>Click to expand</b></summary>

docs/source/deployment/databricks/databricks_ide_development_workflow.md

@@ -184,7 +184,7 @@ Open your newly-created notebook and create **four new cells** inside it. You wi
 1. Before you import and run your Python code, you'll need to install your project's dependencies on the cluster attached to your notebook. Your project has a `requirements.txt` file for this purpose. Add the following code to the first new cell to install the dependencies:
 
 ```ipython
-%pip install -r "/Workspace/Repos/<databricks_username>/iris-databricks/src/requirements.txt"
+%pip install -r "/Workspace/Repos/<databricks_username>/iris-databricks/requirements.txt"
 ```
 
 2. To run your project in your notebook, you must load the Kedro IPython extension. Add the following code to the second new cell to load the IPython extension:

docs/source/deployment/databricks/databricks_notebooks_development_workflow.md

@@ -213,7 +213,7 @@ Create **four new cells** inside your notebook. You will fill these cells with c
 1. Before you import and run your Python code, you'll need to install your project's dependencies on the cluster attached to your notebook. Your project has a `requirements.txt` file for this purpose. Add the following code to the first new cell to install the dependencies:
 
 ```ipython
-%pip install -r "/Workspace/Repos/<databricks_username>/iris-databricks/src/requirements.txt"
+%pip install -r "/Workspace/Repos/<databricks_username>/iris-databricks/requirements.txt"
 ```
 
 2. To run your project in your notebook, you must load the Kedro IPython extension. Add the following code to the second new cell to load the IPython extension:

docs/source/deployment/distributed.md

@@ -14,7 +14,7 @@ For better dependency management, we encourage you to containerise the entire pi
 Firstly make sure your [project requirements are up-to-date](../kedro_project_setup/dependencies.md) by running:
 
 ```bash
-pip-compile --output-file=<project_root>/src/requirements.txt --input-file=<project_root>/src/requirements.txt
+pip-compile --output-file=<project_root>/requirements.txt --input-file=<project_root>/requirements.txt
 ```
 
 We then recommend the [`Kedro-Docker`](https://github.com/kedro-org/kedro-plugins/tree/main/kedro-docker) plugin to streamline the process of building the image. [Instructions for using this are in the plugin's README.md](https://github.com/kedro-org/kedro-plugins/blob/main/README.md).

docs/source/deployment/single_machine.md

@@ -114,7 +114,7 @@ conda install -c conda-forge kedro
 Install the project’s dependencies, by running the following in the project's root directory:
 
 ```console
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 After having installed your project on the remote server you can run the Kedro project as follows from the root of the project:

docs/source/development/commands_reference.md

@@ -273,10 +273,10 @@ def run(
 
 #### Install all package dependencies
 
-The following runs [`pip`](https://github.com/pypa/pip) to install all package dependencies specified in `src/requirements.txt`:
+The following runs [`pip`](https://github.com/pypa/pip) to install all package dependencies specified in `requirements.txt`:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 For further information, see the [documentation on installing project-specific dependencies](../kedro_project_setup/dependencies.md#install-project-specific-dependencies).

docs/source/development/linting.md

@@ -20,7 +20,7 @@ type. [You can read more in the `isort` documentation](https://pycqa.github.io/i
 
 
 ### Install the tools
-Install `black` and `ruff` by adding the following lines to your project's `src/requirements.txt`
+Install `black` and `ruff` by adding the following lines to your project's `requirements.txt`
 file:
 ```text
 black # Used for formatting code
@@ -30,7 +30,7 @@ ruff # Used for linting, formatting and sorting module imports
 To install all the project-specific dependencies, including the linting tools, navigate to the root directory of the
 project and run:
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 Alternatively, you can individually install the linting tools using the following shell commands:
 ```bash
@@ -72,7 +72,7 @@ These hooks are run before committing your code to your repositories to automati
 making code reviews easier and less time-consuming.
 
 ### Install `pre-commit`
-You can install `pre-commit` along with other dependencies by including it in the `src/requirements.txt` file of your
+You can install `pre-commit` along with other dependencies by including it in the `requirements.txt` file of your
 Kedro project by adding the following line:
 ```text
 pre-commit

docs/source/experiment_tracking/index.md

@@ -72,7 +72,7 @@ cd spaceflights
 Install the project's dependencies:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 ## Set up the session store
@@ -268,7 +268,7 @@ In this section, we illustrate how to compare Matplotlib plots across experiment
 
 ### Update the dependencies
 
-Update the `src/requirements.txt` file in your Kedro project by adding the following dataset to enable Matplotlib for your project:
+Update the `requirements.txt` file in your Kedro project by adding the following dataset to enable Matplotlib for your project:
 
 ```text
 kedro-datasets[matplotlib.MatplotlibWriter]~=1.1
@@ -278,7 +278,7 @@ seaborn~=0.12.1
 And install the requirements with:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 ### Add a plotting node

docs/source/get_started/new_project.md

@@ -11,7 +11,7 @@ There are a few ways to create a new project once you have [set up Kedro](instal
 
 Once you've created a project:
 
-* You need to **navigate to its project folder** and **install its dependencies**: `pip install -r src/requirements.txt`
+* You need to **navigate to its project folder** and **install its dependencies**: `pip install -r requirements.txt`
 * **To run the project**: `kedro run`
 * **To visualise the project**: `kedro viz`
 
@@ -79,7 +79,7 @@ kedro new --starter=pandas-iris
 However you create a Kedro project, once `kedro new` has completed, the next step is to navigate to the project folder (`cd <project-name>`) and install dependencies with `pip` as follows:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 Now run the project:

docs/source/kedro_project_setup/dependencies.md

@@ -5,38 +5,38 @@ Both `pip install kedro` and `conda install -c conda-forge kedro` install the co
 When you create a project, you then introduce additional dependencies for the tasks it performs.
 
 ## Project-specific dependencies
-You can specify a project's exact dependencies in the `src/requirements.txt` file to make it easier for you and others to run your project in the future,
+You can specify a project's exact dependencies in the `requirements.txt` file to make it easier for you and others to run your project in the future,
 and to avoid version conflicts downstream. This can be achieved with the help of [`pip-tools`](https://pypi.org/project/pip-tools/).
 To install `pip-tools` in your virtual environment, run the following command:
 ```bash
 pip install pip-tools
 ```
 
-To add or remove dependencies to a project, edit the `src/requirements.txt` file, then run the following:
+To add or remove dependencies to a project, edit the `requirements.txt` file, then run the following:
 
 ```bash
-pip-compile <project_root>/src/requirements.txt --output-file <project_root>/src/requirements.lock
+pip-compile <project_root>/requirements.txt --output-file <project_root>/requirements.lock
 ```
 
 This will [pip compile](https://github.com/jazzband/pip-tools#example-usage-for-pip-compile) the requirements listed in
-the `src/requirements.txt` file into a `src/requirements.lock` that specifies a list of pinned project dependencies
+the `requirements.txt` file into a `requirements.lock` that specifies a list of pinned project dependencies
 (those with a strict version). You can also use this command with additional CLI arguments such as `--generate-hashes`
 to use `pip`'s Hash Checking Mode or `--upgrade-package` to update specific packages to the latest or specific versions.
 [Check out the `pip-tools` documentation](https://pypi.org/project/pip-tools/) for more information.
 
 ```{note}
-The `src/requirements.txt` file contains "source" requirements, while `src/requirements.lock` contains the compiled version of those and requires no manual updates.
+The `requirements.txt` file contains "source" requirements, while `src/requirements.lock` contains the compiled version of those and requires no manual updates.
 ```
 
-To further update the project requirements, modify the `src/requirements.txt` file (not `src/requirements.lock`) and re-run the `pip-compile` command above.
+To further update the project requirements, modify the `requirements.txt` file (not `src/requirements.lock`) and re-run the `pip-compile` command above.
 
 
 ## Install project-specific dependencies
 
 To install the project-specific dependencies, navigate to the root directory of the project and run:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 ## Workflow dependencies

docs/source/nodes_and_pipelines/micro_packaging.md

@@ -10,7 +10,7 @@ You can package a micro-package by executing: `kedro micropkg package <micropkg_
 
 `kedro micropkg package pipelines.data_processing`
 
-* This will generate a new [source distribution](https://docs.python.org/3/distutils/sourcedist.html) for this micro-package.
+* This will generate a new [source distribution](https://docs.python.org/3.11/distutils/sourcedist.html) for this micro-package.
 * By default, the tar file will be saved into `dist/` directory inside your project.
 * You can customise the target with the `--destination` (`-d`) option.
 

docs/source/tutorial/spaceflights_tutorial_faqs.md

@@ -46,7 +46,7 @@ documentation on how to install relevant dependencies for kedro_datasets.pandas.
 https://kedro.readthedocs.io/en/stable/kedro_project_setup/dependencies.html
 ```
 
-The Kedro Data Catalog is missing [dependencies needed to parse the data](../kedro_project_setup/dependencies.md#install-dependencies-related-to-the-data-catalog). Check that you have [all the project dependencies to `requirements.txt`](./tutorial_template.md#install-project-dependencies) and then call `pip install -r src/requirements.txt` to install them.
+The Kedro Data Catalog is missing [dependencies needed to parse the data](../kedro_project_setup/dependencies.md#install-dependencies-related-to-the-data-catalog). Check that you have [all the project dependencies to `requirements.txt`](./tutorial_template.md#install-project-dependencies) and then call `pip install -r requirements.txt` to install them.
 
 ### Pipeline run
 

docs/source/tutorial/tutorial_template.md

@@ -1,6 +1,6 @@
 # Set up the spaceflights project
 
-This section shows how to create a new project (with `kedro new` using the [Kedro spaceflights starter](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights)) and install project dependencies (with `pip install -r src/requirements.txt`).
+This section shows how to create a new project (with `kedro new` using the [Kedro spaceflights starter](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights)) and install project dependencies (with `pip install -r requirements.txt`).
 
 ## Create a new project
 
@@ -28,7 +28,7 @@ cd spaceflights
 
 Kedro projects have a `requirements.txt` file to specify their dependencies and enable sharable projects by ensuring consistency across Python packages and versions.
 
-The spaceflights project dependencies are stored in `src/requirements.txt`(you may find that the versions differ slightly depending on the version of Kedro):
+The spaceflights project dependencies are stored in `requirements.txt`(you may find that the versions differ slightly depending on the version of Kedro):
 
 ```text
 # code quality packages
@@ -60,7 +60,7 @@ scikit-learn~=1.0
 To install all the project-specific dependencies, run the following from the project root directory:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 ## Optional: logging and configuration

docs/source/visualisation/kedro-viz_visualisation.md

@@ -18,7 +18,7 @@ When prompted for a project name, you can enter anything, but we will assume `Sp
 When your project is ready, navigate to the root directory of the project and install the dependencies for the project, which include Kedro-Viz:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 The next step is optional, but useful to check that all is working. Run the full set of pipelines for the tutorial project:

docs/source/visualisation/visualise_charts_with_plotly.md

@@ -38,7 +38,7 @@ kedro-datasets[plotly.PlotlyDataSet, plotly.JSONDataSet]~=1.1
 Navigate to the root directory of the project in your terminal and install the dependencies for the tutorial project:
 
 ```bash
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 ### Configure the Data Catalog
@@ -177,7 +177,7 @@ You can view Matplotlib charts in Kedro-Viz when you use the [Kedro MatplotLibWr
 
 ### Update the dependencies
 
-You must update the `src/requirements.txt` file in the Kedro project by adding the following dataset to enable Matplotlib for the project:
+You must update the `requirements.txt` file in the Kedro project by adding the following dataset to enable Matplotlib for the project:
 
 ```bash
 kedro-datasets[matplotlib.MatplotlibWriter]~=1.1

features/environment.py

@@ -113,9 +113,7 @@ def _setup_minimal_env(context):
 
 def _install_project_requirements(context):
     install_reqs = (
-        Path(
-            "kedro/templates/project/{{ cookiecutter.repo_name }}/src/requirements.txt"
-        )
+        Path("kedro/templates/project/{{ cookiecutter.repo_name }}/requirements.txt")
         .read_text(encoding="utf-8")
         .splitlines()
     )

features/steps/cli_steps.py

@@ -163,7 +163,7 @@ def create_config_file(context):
 @given("I have installed the project dependencies")
 def pip_install_dependencies(context):
     """Install project dependencies using pip."""
-    reqs_path = "src/requirements.txt"
+    reqs_path = "requirements.txt"
     res = run(
         [context.pip, "install", "-r", reqs_path],
         env=context.env,
@@ -409,7 +409,7 @@ def update_pyproject_toml(context: behave.runner.Context, new_source_dir):
 @given("I have updated kedro requirements")
 def update_kedro_req(context: behave.runner.Context):
     """Remove kedro as a standalone requirement."""
-    reqs_path = context.root_project_dir / "src" / "requirements.txt"
+    reqs_path = context.root_project_dir / "requirements.txt"
 
     if reqs_path.is_file():
         old_reqs = reqs_path.read_text().splitlines()
@@ -427,7 +427,7 @@ def update_kedro_req(context: behave.runner.Context):
 
 @when("I add {dependency} to the requirements")
 def add_req(context: behave.runner.Context, dependency: str):
-    reqs_path = context.root_project_dir / "src" / "requirements.txt"
+    reqs_path = context.root_project_dir / "requirements.txt"
     if reqs_path.is_file():
         reqs_path.write_text(reqs_path.read_text() + "\n" + str(dependency) + "\n")
 
@@ -605,14 +605,14 @@ def check_docs_generated(context: behave.runner.Context):
 @then("requirements should be generated")
 def check_reqs_generated(context: behave.runner.Context):
     """Check that new project requirements are generated."""
-    reqs_path = context.root_project_dir / "src" / "requirements.lock"
+    reqs_path = context.root_project_dir / "requirements.lock"
     assert reqs_path.is_file()
     assert "This file is autogenerated by pip-compile" in reqs_path.read_text()
 
 
 @then("{dependency} should be in the requirements")
 def check_dependency_in_reqs(context: behave.runner.Context, dependency: str):
-    reqs_path = context.root_project_dir / "src" / "requirements.txt"
+    reqs_path = context.root_project_dir / "requirements.txt"
     assert dependency in reqs_path.read_text()
 
 

features/steps/test_starter/{{ cookiecutter.repo_name }}/README.md

@@ -17,12 +17,12 @@ In order to get the best out of the template:
 
 ## How to install dependencies
 
-Declare any dependencies in `src/requirements.txt` for `pip` installation.
+Declare any dependencies in `requirements.txt` for `pip` installation.
 
 To install them, run:
 
 ```
-pip install -r src/requirements.txt
+pip install -r requirements.txt
 ```
 
 ## How to run Kedro
@@ -46,7 +46,7 @@ To configure the coverage threshold, look at the `.coveragerc` file.
 
 ## Project dependencies
 
-To see and update the dependency requirements for your project use `src/requirements.txt`. You can install the project requirements with `pip install -r src/requirements.txt`.
+To see and update the dependency requirements for your project use `requirements.txt`. You can install the project requirements with `pip install -r requirements.txt`.
 
 [Further information about project dependencies](https://docs.kedro.org/en/stable/kedro_project_setup/dependencies.html#project-specific-dependencies)
 

features/steps/test_starter/{{ cookiecutter.repo_name }}/pyproject.toml

@@ -1,7 +1,40 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "{{ cookiecutter.python_package }}"
+readme = "README.md"
+dynamic = ["dependencies", "version"]
+
+[project.scripts]
+{{ cookiecutter.repo_name }} = "{{ cookiecutter.python_package }}.__main__:main"
+
+[project.optional-dependencies]
+docs = [
+    "docutils<0.18.0",
+    "sphinx~=3.4.3",
+    "sphinx_rtd_theme==0.5.1",
+    "nbsphinx==0.8.1",
+    "sphinx-autodoc-typehints==1.11.1",
+    "sphinx_copybutton==0.3.1",
+    "ipykernel>=5.3, <7.0",
+    "Jinja2<3.1.0",
+    "myst-parser~=0.17.2",
+]
+
+[tool.setuptools.dynamic]
+dependencies = {file = "requirements.txt"}
+version = {attr = "{{ cookiecutter.python_package }}.__version__"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+namespaces = false
+
 [tool.kedro]
-project_name = "{{ cookiecutter.project_name }}"
-project_version = "{{ cookiecutter.kedro_version }}"
 package_name = "{{ cookiecutter.python_package }}"
+project_name = "{{ cookiecutter.project_name }}"
+kedro_init_version = "{{ cookiecutter.kedro_version }}"
 
 [tool.pytest.ini_options]
 addopts = """