Refactor main methods for scripts to run without snakemake

[fhg-isi]

I tried to run

https://github.com/PyPSA/pypsa-eur/blob/master/scripts/prepare_sector_network.py

as python script to debug and learn about it.

I got an error due to non-existing config file

test/config.overnight.yaml

After changing it to

config/test/config.overnight.yaml

the command snakemake = mock_snakemake(..) worked.

Edit
Current version does not include the non-existing config file but has it out-commented

# configfiles="test/config.overnight.yaml",

I suggest to :

a) Check all main functions of the scripts if they work without "snakemake" in globals().
(Maybe automatically run them in a github workflow if possible.)

Edit: There is already a CI workflow to run test.sh which calls the rule solve_elec_networks. Currently the pipeline fails.
Maybe extend workflow with rule solve_sector_networks?

b) Update or remove paths to config files for calls to mock_snakemake.

c) Extract main content starting after line if __name__ == "__main__" to an extra function def main(..): and move it to the top of the file.

d) Improve handling of missing input files to make it easier for beginners. Here are some ideas:

• Warning/Hint about missing input: Include checks for missing input files, for example (might not work; just to illustrate):

output_path = snakemake.output[0]
network_path = snakemake.input.network
if not os.path.exists(network_path):
      message = f'Network file {network_path} does not exist yet. ' +\
      'Snakemake command to create it: ' + \
        f'snakemake --cores 1 {network_path}' +\
     'Snakemake command to run full skript and auto-generate required inputs' +\
      f'snakemake --cores 1 {output_path }' 
      raise KeyError(message)

• Adapt mock_snakemake to automatically try to generate input files, if they do not exist yet.

e) Consider to provide a consistent set of pre-existing input files for a "small" example, that allows
to run each of the rule immediately with default settings for macksnakemake... so that users do not
need to generate all the inputs beforehand. That would allow for a quicker entry into debugging/learning.
Furhtermore, if there is a bug in a previous step, it would not block the whole chain.

Related:

• Tutorial example Sector-Coupled does not run through: KeyError: "None of [Index(['diameter_mm', 'max_cap_M_m3_per_d'], dtype='object')] are in the [columns]" #1121
• • Use imports and run directive instead of script directive for Snakefiles / clarify how to debug Snakemake scripts #1122
• https://stackoverflow.com/questions/78677266/how-to-debug-scripts-for-pypsa-eur
• How to debug snakemake files and referenced python scripts in IDEs like PyCharm or VsCode? snakemake/snakemake#2932
• Refactor scripts to follow concept of "one level of abstraction per function"? #1119
• Introduce mocksnakemake which acutally parses Snakefile #107

[fhg-isi]

Please chang from bug to enhancement. (First I thought I found a bug but used an older version. Later I was not able to convert the ticket type from bug to feature request/enhancement.)