Output curves (#70)

* Export output curves to a separate file by default on to_excel

* Swap from pipenv to poetry

* Clean up Jupyter notebooks

Author: louispt1

.gitignore

@@ -18,3 +18,5 @@ import_graph.svg
 tmp/
 
 *.xlsx
+# Keep the example input
+!examples/example_input_excel.xlsx

Pipfile

@@ -1,80 +1,143 @@
 # pyetm
 
-This package provides a set of tools for interaction with the Energy Transition Model's API. Learn more
-about the Energy Transition Model [here](https://energytransitionmodel.com/). The
-package is designed to be a modular tool that advanced users can incorporate into their workflows. The
-complete documentation is available [via the ETM documentation page](https://docs.energytransitionmodel.com/main/pyetm/introduction).
+This package provides a set of tools for interaction with the Energy Transition Model's API.
+Learn more about the Energy Transition Model [here](https://energytransitionmodel.com/).
+
+The package is designed to be a modular tool that advanced users can incorporate into their scenario workflows.
+The complete documentation is available [via the ETM documentation page](https://docs.energytransitionmodel.com/main/pyetm/introduction).
+
+---
 
 ## Installation
 
-You can clone the pyetm from [our Github](https://github.com/quintel/pyetm). The package is also
-available via pip like any other python package - install it and use it in your project!
-```
+You can install **pyetm** directly from PyPI:
+```bash
 pip install pyetm
 ```
 
-## Just running the Jupyter Notebooks - or a beginner friendly guide to PyETM
-If you are not planning on developing the tool, but would like to open and run our Jupyter notebooks
-in VS Code, follow the beginner friendly guide at [Running notebooks](running_notebooks.md).
+Or clone from [our GitHub repository](https://github.com/quintel/pyetm) if you want the latest development version:
+```bash
+git clone https://github.com/quintel/pyetm.git
+cd pyetm
+```
 
-## Getting started
-Make sure you have [Python 3](https://www.python.org/downloads/) installed. Then, install all required
-libraries by opening a terminal/command-prompt window in the `pyetm` folder (or navigate to this folder
-in the terminal using `cd "path/to/scenario-tools-folder"`). All following examples of running the tool
-expect you to be in this folder.
+---
 
-#### Using pipenv
-It is recommended (but not required) that you use [`pipenv`](https://pipenv.pypa.io/en/latest/) for
-running these tools. When using `pipenv` it will create a virtual environment for you. A virtual
-environment helps with keeping the libraries you install here separate of your global libraries (in
-other words your `pyetm` will be in a stable and isolated environment and are thus less
-likely to break when updating things elsewhere on your computer).
+## Running Jupyter Notebooks (Beginner Friendly)
 
-You can install `pipenv` with `pip` or `pip3` if you don't have it installed yet.
-```
-pip3 install pipenv
-```
+If you only want to open and run our Jupyter notebooks in VS Code without developing the package,
+follow the beginner guide here: [Running notebooks](running_notebooks.md).
 
-Then you can create a new environment and install all the libraries in one go by running:
-```
-pipenv install
-```
+---
 
-If you plan to develop with the tool, install the dev dependencies too:
-```
-pipenv install --dev
+## Development Setup (Using Poetry)
+
+We recommend using [Poetry](https://python-poetry.org/) to manage dependencies and virtual environments.
+Poetry ensures all dependencies are installed in an isolated environment, keeping your system clean.
+
+### Python
+Make sure you have **Python 3.12** or later installed:
+- **Windows**: [Download from python.org](https://www.python.org/downloads/windows/)
+- **macOS**: Install via [Homebrew](https://brew.sh/)
+  ```bash
+  brew install python@3.12
+  ```
+- **Linux**: Use your package manager or install from source.
+
+Check your version:
+```bash
+python3 --version
 ```
 
-#### Configuring your settings
+---
 
-You can set your API token and the base url for your requests (depending which
-[environment](https://docs.energytransitionmodel.com/api/intro#environments) you want to interact with)
-either directly in the ENV or via a config.yml file.
+### Poetry
+Follow the [official instructions](https://python-poetry.org/docs/#installation):
 
-##### config.yml
-pyetm uses a `config.yml` file in the project root to store your personal settings:
+```bash
+curl -sSL https://install.python-poetry.org | python3 -
+```
 
-1. Duplicate the example file provided (`examples/config.example.yml`) and rename it to `config.yml`.
-2. Open `config.yml` and fill in your values:
-   - **etm_api_token**: Your personal ETM API token (overridden by the `$ETM_API_TOKEN` environment variable if set).
-   - **base_url**: The API base URL for the target environment (overridden by the `$BASE_URL` environment
-    variable if set) e.g., default pro, a stable engine at `https://2025-01.engine.energytransitionmodel.com/api/v3`,
-    or beta at `https://beta.engine.energytransitionmodel.com/api/v3`.
-   - **local_engine_url** and **local_model_url**: URLs for a local ETM instance, if running locally.
-   - **proxy_servers**: (Optional) HTTP/HTTPS proxy URLs, if required by your network.
-   - **csv_separator** and **decimal_separator**: Defaults are `,` and `.`; adjust if your CSV exports
-    use different separators.
+After installation, ensure Poetry is available:
+```bash
+poetry --version
+```
 
-Your `config.yml` should reside in the root `pyetm/` folder.
 
-##### ENV variables
-If you use pyetm as a package, you may want to set your ENV variables using a custom flow. In that
-case, the variables you need to set are:
+#### Install Dependencies
 
-    $ETM_API_TOKEN - Your api token (specific to the environment you are interacting with)
-    $BASE_URL - The base url of the environment you are interacting with.
-    $LOCAL_ENGINE_URL - The local url of the engine if running locally.
-    $LOCAL_MODEL_URL - The local url of the model if running locally.
+Navigate to the `pyetm` folder and install all dependencies:
+```bash
+cd pyetm
+poetry install
+```
 
+This will:
+- Create a virtual environment
+- Install runtime dependencies
+If you want development dependencies (testing, linting, etc.) then append the
+"--with dev" flag to the install command.
+
+
+#### Activating the Environment
+You can either:
+- Run commands inside Poetry’s environment:
+  ```bash
+  poetry run pytest
+  poetry run pyetm
+  ```
+- Or activate the shell:
+  ```bash
+  eval $(poetry env activate)
+  ```
+  Then run commands normally:
+  ```bash
+  pytest
+  pyetm
+  ```
+
+
+## Configuring Your Settings
+
+You can configure your API token and base URL either with a **config.yml** file or environment variables. You can now simply set an `environment` and the base URL will be inferred for you.
+
+### Option 1: `config.yml`
+1. Duplicate the example file (`examples/example.config.yml`) and rename it to `config.yml`.
+2. Edit `config.yml`:
+   - **etm_api_token**: Your ETM API token (overridden by `$ETM_API_TOKEN` if set).
+   - **environment**: pro (default), beta, local, or a stable tag like `2025-01`. When set, `base_url` is inferred automatically.
+   - (optional) **base_url**: API base URL (overridden by `$BASE_URL` if set). If both `environment` and `base_url` are set, `base_url` wins.
+     Examples if you need a direct override:
+       - `https://engine.energytransitionmodel.com/api/v3` (pro)
+       - `https://beta.engine.energytransitionmodel.com/api/v3` (beta)
+       - `https://2025-01.engine.energytransitionmodel.com/api/v3` (stable tag)
+   - **proxy_servers**: (Optional) HTTP/HTTPS proxy URLs.
+   - **csv_separator** and **decimal_separator**: Defaults are `,` and `.`.
+
+Place `config.yml` in the project root (`pyetm/` folder).
+
+### Option 2: Environment Variables
+If you prefer, set these environment variables:
+```bash
+ETM_API_TOKEN=<your token>
+ENVIRONMENT=<pro|beta|local|YYYY-MM>
+# or provide a direct override instead of ENVIRONMENT
+BASE_URL=<api url>
+```
 
-#TODO - check links
+---
+
+## Cross-Platform Notes
+- **Windows**:
+  - Use `py` instead of `python3` if `python3` is not recognized.
+  - In PowerShell, set environment variables with:
+    ```powershell
+    $env:ETM_API_TOKEN="your-token"
+    ```
+- **macOS/Linux**:
+  - Use `python3` in commands.
+  - Set environment variables with:
+    ```bash
+    export ETM_API_TOKEN="your-token"
+  export ENVIRONMENT=beta   # or pro/local/2025-01
+    ```