Improve onboarding instructions for PyCharm/IntelliJ users

contributing-docs/quick-start-ide/contributors_quick_start_pycharm.rst

@@ -29,7 +29,6 @@ Setup your project
              alt="Cloning github fork to Pycharm">
       </div>
 
-
 2. Paste the repository link in the URL field and submit.
 
    .. raw:: html
@@ -39,52 +38,37 @@ Setup your project
              alt="Cloning github fork to Pycharm">
       </div>
 
-3. Configure the source root directories well as for ``task-sdk`` and ``devel-common``.
-   You also have to set "source" and "tests" root directories for each provider you want to develop (!).
+3. Synchronize local ``.venv`` virtualenv using uv
 
-   This is important in Airflow 3.0 we split ``task-sdk``, ``devel-common`` and each provider to be separate
-   distribution - each with separate ``pyproject.toml`` file, so you need to separately
-   add "src" and "tests" directories for each provider you develop to be respectively
-   "source roots" and "test roots".
+    .. code-block:: bash
 
-   This might be improve and might be better automated in the future, but for now you need to do it
-   for each provider separately.
+      $ uv sync
 
-   .. raw:: html
+This will create ``.venv`` virtual environment in the project root directory and install all the dependencies of
+airflow core. If you plan to work on providers, at this time you can install dependencies for all providers:
 
-      <div align="center" style="padding-bottom:10px">
-        <img src="images/pycharm_add_provider_sources_and_tests.png"
-             alt="Adding Source Root directories to Pycharm">
-      </div>
+    .. code-block:: bash
 
-   You also need to add ``task-sdk`` sources (and ``devel-common`` in similar way).
+      $ uv sync --all-packages
 
-   .. raw:: html
+Or for specific provider and it's cross-provider dependencies:
 
-      <div align="center" style="padding-bottom:10px">
-        <img src="images/pycharm_add_task-sdk_sources.png"
-             alt="Adding Source Root directories to Pycharm">
-      </div>
-
-4. Once step 3 is done it is recommended to invalidate caches and restart Pycharm.
+    .. code-block:: bash
 
-   .. raw:: html
+      $ uv sync --packages apache-airflow-provider-amazon
 
-      <div align="center" style="padding-bottom:10px">
-        <img src="images/pycharm_invalidate_caches.png"
-             alt="Invalidate caches and restart Pycharm">
-      </div>
+Next: Configure your IDEA project.
 
-5. An alternative way to add source roots is to configure the ``airflow.iml`` file under ``.idea`` directory and update the
-   ``module.xml`` file:
+3. The fastest way to add source roots is to configure the ``airflow.iml`` file under ``.idea`` directory and update the
+   ``module.xml`` file using the ``setup_idea.py`` script:
 
    To setup the source roots for all the modules that exist in the project, you can run the following command:
    This needs to done on the airflow repository root directory. It overwrites the existing ``.idea/airflow.iml`` and
-   ``.idea/modules.xml`` files.
+   ``.idea/modules.xml`` files if they exist.
 
     .. code-block:: bash
 
-      $ python setup_idea.py
+      $ uv run setup_idea.py
 
    Then Restart the PyCharm/IntelliJ IDEA.
 
@@ -102,6 +86,52 @@ Setup your project
               alt="modules.xml">
         </div>
 
+4. Alternatively, you can configure your project manually. Configure the source root directories well
+   as for ``airflow-core`` ``task-sdk``, ``airflow-ctl`` and ``devel-common``. You also have to set
+   "source" and "tests" root directories for each provider you want to develop (!).
+
+   In Airflow 3.0 we split ``airflow-core``, ``task-sdk``, ``airflow-ctl``, ``devel-common``,
+   and each provider to be separate distribution - each with separate ``pyproject.toml`` file,
+   so you need to separately add ``src`` and ``tests`` directories for each provider you develop
+   to be respectively "source roots" and "test roots".
+
+   .. raw:: html
+
+      <div align="center" style="padding-bottom:10px">
+        <img src="images/pycharm_add_provider_sources_and_tests.png"
+             alt="Adding Source Root directories to Pycharm">
+      </div>
+
+   You also need to add ``task-sdk`` sources (and ``devel-common`` in similar way).
+
+   .. raw:: html
+
+      <div align="center" style="padding-bottom:10px">
+        <img src="images/pycharm_add_task-sdk_sources.png"
+             alt="Adding Source Root directories to Pycharm">
+      </div>
+
+5. Once step 3 or 4 is done you should configure python interpreter for your PyCharm/IntelliJ to use
+   the virtualenv created by ``uv sync``.
+
+    .. raw:: html
+
+        <div align="center" style="padding-bottom:10px">
+          <img src="images/pycharm_add_interpreter.png"
+              alt="Configuring Python Interpreter">
+        </div>
+
+6. It is recommended to invalidate caches and restart PyCharm after setting up the project.
+
+   .. raw:: html
+
+      <div align="center" style="padding-bottom:10px">
+        <img src="images/pycharm_invalidate_caches.png"
+             alt="Invalidate caches and restart Pycharm">
+      </div>
+
+
+
 Setting up debugging
 ####################
 

setup_idea.py

@@ -14,9 +14,18 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
+# /// script
+# requires-python = ">=3.9"
+# dependencies = [
+#   "rich>=13.6.0",
+# ]
+# ///
 from __future__ import annotations
 
-import os
+from pathlib import Path
+
+from rich import print
+from rich.prompt import Confirm
 
 iml_xml_template = """<?xml version="1.0" encoding="UTF-8"?>
 <module type="PYTHON_MODULE" version="4">
@@ -57,43 +66,52 @@
 
 source_root_module_patter: str = '<sourceFolder url="file://$MODULE_DIR$/{path}" isTestSource="{status}" />'
 
-source_root_modules: list[str] = ["airflow-core", "airflow-ctl", "dev/breeze", "task-sdk"]
+source_root_modules: list[str] = ["airflow-core", "airflow-ctl", "task-sdk", "devel-common", "dev/breeze"]
 
 all_module_paths: list[str] = []
 
-ROOT_DIR: str = "./providers"
+ROOT_AIRFLOW_FOLDER_PATH = Path(__file__).parent
+IDEA_FOLDER_PATH = ROOT_AIRFLOW_FOLDER_PATH / ".idea"
+AIRFLOW_IML_FILE = IDEA_FOLDER_PATH / "airflow.iml"
+MODULES_XML_FILE = IDEA_FOLDER_PATH / "modules.xml"
+
+ROOT_PROVIDERS_FOLDER_PATH = ROOT_AIRFLOW_FOLDER_PATH / "providers"
 
 
 def setup_idea():
     # Providers discovery
-    for dirpath, _, filenames in os.walk(ROOT_DIR):
-        if "pyproject.toml" in filenames:
-            relative_path = os.path.relpath(dirpath, ROOT_DIR)
-            source_root_modules.append(f"providers/{relative_path}")
+    for pyproject_toml_file in ROOT_PROVIDERS_FOLDER_PATH.rglob("pyproject.toml"):
+        relative_path = pyproject_toml_file.relative_to(ROOT_AIRFLOW_FOLDER_PATH).parent.as_posix()
+        source_root_modules.append(f"{relative_path}")
 
     source_root_modules.sort()
     for module in source_root_modules:
+        print(f"[green]Adding[/] module: [blue]{module}[/]")
         all_module_paths.append(source_root_module_patter.format(path=f"{module}/src", status="false"))
-        all_module_paths.append(source_root_module_patter.format(path=f"{module}/test", status="true"))
-
-    source_root_module_path: str = "\n\t\t".join(all_module_paths)
+        all_module_paths.append(source_root_module_patter.format(path=f"{module}/tests", status="true"))
 
-    base_source_root_xml: str = iml_xml_template.format(SOURCE_ROOT_MODULE_PATH=source_root_module_path)
+    source_root_module_path = "\n\t\t".join(all_module_paths)
 
-    with open(".idea/airflow.iml", "w") as file:
-        file.write(base_source_root_xml)
+    base_source_root_xml = iml_xml_template.format(SOURCE_ROOT_MODULE_PATH=source_root_module_path)
 
-    with open(".idea/modules.xml", "w") as file:
-        file.write(module_xml_template)
+    IDEA_FOLDER_PATH.mkdir(exist_ok=True)
+    AIRFLOW_IML_FILE.write_text(base_source_root_xml)
+    MODULES_XML_FILE.write_text(module_xml_template)
 
 
 if __name__ == "__main__":
-    user_input = input(
-        "This script will overwrites the .idea/airflow.iml and .idea/modules.xml files. Press Enter Y/N to continue: "
-    )
-    if user_input.lower() == "y":
+    print("\n[yellow]Warning!!![/] This script will update the PyCharm/IntelliJ IDEA configuration files:\n")
+    print(f"* {AIRFLOW_IML_FILE}")
+    print(f"* {MODULES_XML_FILE}\n")
+    should_continue = Confirm.ask("Overwrite the files?")
+    if should_continue:
+        print()
         setup_idea()
-        print("Updated airflow.iml and modules.xml files, Now restart the PyCharm/IntelliJ IDEA")
+        print("\n[green]Success\n")
+        print(
+            f"Updated {AIRFLOW_IML_FILE} and {MODULES_XML_FILE} files. "
+            f"Now restart the PyCharm/IntelliJ IDEA\n"
+        )
     else:
-        print("Not updating airflow.iml and modules.xml files")
-        exit(0)
+        print("[yellow]Skipped\n")
+        print(f"Not updated {AIRFLOW_IML_FILE} and {MODULES_XML_FILE} files\n")