UI for schedule and ship config building

[j-atkins]

Here's a version of the new UI tool for interactive schedule and ship config building. Inevitably there will be bits that can be improved, so I look forward to the feedback. See GIF at the bottom for quick look at the UI in action!

This is quite a large PR (apologies...) but I'll give an overview the new proposed workflow next. And then also some notes on the UI's features and some notes/considerations.

Proposed workflow

This PR adds a new (optional) virtualship plan command to the CLI. This command is to be used after virtualship init.

So, the workflow is now something like:

1. virtualship init [EXPEDITION_DIR] to generate the 'schedule.yaml' and 'ship_config.yaml' files.
2. virtualship plan [EXPEDITION_DIR] reads in the newly generated YAMLs, allows the user to modify values in both (including waypoint locations, timings and instrument selections) before writing the updated entries to the YAMLs on pressing the save button.
3. virtualship fetch [EXPEDITION_DIR] as before.
4. virtualship run [EXPEDITION_DIR] as before.

UI features

• Input boxes in the UI will not allow invalid input types. For example, the user is prevented from typing anything other than numbers into a box expecting a float (e.g. lat/lon inputs). Where possible the requirements are determined from the respective Pydantic models.
• There is direct messaging to the user if the input is invalid. For example, a latitude input must be -90 <= lat <= 90, time periods cannot be negative. Where possible the requirements are determined from the respective Pydantic models.
• schedule.verify() methods are run upon pressing save, checking that the scheduling is valid.
• Error messaging: I've tried to add informative error messaging for the user (including new UserError and UnexpectedError classes). Depending on the error, users will be notified by messages within the UI or in the terminal log if the error is caught on launch. In cases where the error is unexpected, users are prompted to raise an issue on the VirtualShip issue tracker.
• New tests which simulate pressing different buttons in the UI, changing inputs etc. The saving procedure is also simulated and checks that the save was successful and that the YAML has been updated with the new choices.

Notes

• New textual and pytest-asyncio dependencies added.
• The UI is optional, so if users prefer to interact directly with the YAMLs they can still of course.
• Now, if using virtualship init with the --from-mfp option, this will no longer configure the instrument choices from the MFP export file. This therefore removes the previous method where users would add an extra Instrument column in the Excel file. This step is now handled in the UI.
• start_time and end_time in 'ship_config.yaml' are set up to be autofilled if they are left blank in the UI (given they fall under the "advanced users only" section in the UI). It's not immediately clear to me what end_time should automatically default to. I have currently set it to take the time of the last waypoint + 42 days to cover the default drifter lifetime in case a drifter is selected in the last waypoint. However, I also notice that the drifter data download in fetch adds a 21-day buffer. I'm not sure where it's best to handle this.
• When selecting the waypoint times there are drop-downs in the UI, which helps make sure only valid entries are selected. Currently the 'Year' dropdown is hard-coded to start from 2022 to present (reflecting the fact that VirtualShip currently only supports use with the Copernicus Analysis & Forecast products). I think some thought might be needed here on how to better handle this in the situation that we start adding more data sources ... do we present a much longer list of years and rely on in-class messaging and/or the documentation to explain which can be selected depending on the data source? Or rely on the fact that the invalid choices will inevitably be picked up when it comes to fetching the data?
• I have experimented with using textual-serve for launching the UI in browser (as discussed with @erikvansebille previously). However, given the focus on potential future cloud integration (e.g. EDITO) where VirtualShip would be run in browser-based JupyterLab terminals I favour, for now, keeping it as a terminal-based tool which launches in the JupyterLab terminal rather than having to deal with launching to browser from the browser. I have checked and the UI still looks nice when run in a JupyterLab terminal. That being said, I'm open to adding browser launch support in a future PR if we think it would be valuable...
• TODO: documentation (including the new quickstart guide, i.e. New quickstart guide #192) will need updating soon!

Quite a lot of text there but let me know if anything's unclear!

P.S. GIF!

[erikvansebille]

Very nice, @j-atkins! I played around a bit and it worked quite smooth! A few minor user-oriented comments below

[erikvansebille]

DO we need to explain a bit more what these instruments are? Or where to find more information about them?

[erikvansebille]

Make clear in the tool that lifetime is in minutes? And why not days, like Argo?

[erikvansebille]

The (expected) error I see now is that there is an error saving changes, each waypoint should be after another. But would it help users to notify at which waypoint the error happens?

[erikvansebille]

Is there also a way to delete waypoints in the tool?

[VeckoTheGecko]

(or add)

[VeckoTheGecko]

Just a couple points of clarification re. data model and UI. I haven't done an in depth review of the code itself in _plan.py (since this is an isolated part of the codebase, and I'm not familiar with the conventions used in textualised code) - but have briefly looked through it and I think its good :)

[VeckoTheGecko]

I think that this should be between -180 to 360 (i.e., in line with copernicusmarine)until we decide within virtualship exactly how we are handling crossing the international date line

[VeckoTheGecko]

(or add)

[j-atkins]

Thanks for the reviews - here are the requested changes!

In summary:

• Longitude validation requirements revered back to between -180 and 360.
• I have added a "?" button which explains the difference between OceanObserver and SeaSeven in the ADCP selection.
• Re: ‘error saving changes, each waypoint should be after another’, this now specifies which waypoints.
• There are now add and remove waypoint buttons.
  • It would also be nice to support re-ordering the waypoints but this gets a bit complicated and also leads to more questions over how you handle the fact that re-ordering the whole collapsible for each waypoint will often invalidate the schedule because of the wrong time order.

In addition:

• I have added ability to choose how many drifters to deploy per waypoint
  • However, I notice when all drifters are deployed at the exact same lat/lon and time (i.e. the same waypoint) they will all move deterministically with the Eulerian flow. I know introducing stochastic processes/diffusion is would be simple enough to introduce with Parcels but just wanted to flag now in case I’m out of the loop and this is something that has been thought about previously in the context of drifters in VirtualShip? Or even if there’s a particular reason we wouldn't want to introduce this/hasn't already?

[VeckoTheGecko]

I've been encountering a bug with adding and removing waypoints where it crashes if you quickly press remove waypoints. (to recreate you can quickly add a bunch, and quickly remove a bunch). Do you know what could be the cause of that @j-atkins ?

(ship-ui) 🦎ship   ui-schedule[$?] ❯ virtualship plan test
╭─────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────╮
│ /Users/Hodgs004/coding/repos/ship/src/virtualship/cli/_plan.py:514 in show_invalid_reasons                                                       │
│                                                                                                                                                  │
│    511 │   def show_invalid_reasons(self, event: Input.Changed) -> None:                                                                         │
│    512 │   │   input_id = event.input.id                                                                                                         │
│    513 │   │   label_id = f"validation-failure-label-{input_id}"                                                                                 │
│ ❱  514 │   │   label = self.query_one(f"#{label_id}", Label)                                                                                     │
│    515 │   │   if input_id.endswith("_drifter_count"):                                                                                           │
│    516 │   │   │   wp_index = int(input_id.split("_")[0][2:])                                                                                    │
│    517 │   │   │   drifter_switch = self.query_one(f"#wp{wp_index}_DRIFTER")                                                                     │
│                                                                                                                                                  │
│ ╭─────────────────── locals ────────────────────╮                                                                                                │
│ │    event = Changed()                          │                                                                                                │
│ │ input_id = 'wp5_lat'                          │                                                                                                │
│ │ label_id = 'validation-failure-label-wp5_lat' │                                                                                                │
│ │     self = ScheduleEditor()                   │                                                                                                │
│ ╰───────────────────────────────────────────────╯                                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
NoMatches: No nodes match '#validation-failure-label-wp5_lat' on ScheduleEditor()
(ship-ui) 🦎ship   ui-schedule[$?] ❯ 

[j-atkins]

I've been encountering a bug with adding and removing waypoints where it crashes if you quickly press remove waypoints. (to recreate you can quickly add a bunch, and quickly remove a bunch). Do you know what could be the cause of that @j-atkins ?

(ship-ui) 🦎ship   ui-schedule[$?] ❯ virtualship plan test
╭─────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────╮
│ /Users/Hodgs004/coding/repos/ship/src/virtualship/cli/_plan.py:514 in show_invalid_reasons                                                       │
│                                                                                                                                                  │
│    511 │   def show_invalid_reasons(self, event: Input.Changed) -> None:                                                                         │
│    512 │   │   input_id = event.input.id                                                                                                         │
│    513 │   │   label_id = f"validation-failure-label-{input_id}"                                                                                 │
│ ❱  514 │   │   label = self.query_one(f"#{label_id}", Label)                                                                                     │
│    515 │   │   if input_id.endswith("_drifter_count"):                                                                                           │
│    516 │   │   │   wp_index = int(input_id.split("_")[0][2:])                                                                                    │
│    517 │   │   │   drifter_switch = self.query_one(f"#wp{wp_index}_DRIFTER")                                                                     │
│                                                                                                                                                  │
│ ╭─────────────────── locals ────────────────────╮                                                                                                │
│ │    event = Changed()                          │                                                                                                │
│ │ input_id = 'wp5_lat'                          │                                                                                                │
│ │ label_id = 'validation-failure-label-wp5_lat' │                                                                                                │
│ │     self = ScheduleEditor()                   │                                                                                                │
│ ╰───────────────────────────────────────────────╯                                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
NoMatches: No nodes match '#validation-failure-label-wp5_lat' on ScheduleEditor()
(ship-ui) 🦎ship   ui-schedule[$?] ❯ 

Good spot! I'll look into it.

[j-atkins]

Latest commit should now catch and avoid the NoMatches issue when rapidly adding/removing waypoints. Simply returning when there's no matches should mean the UI doesn't crash when the event handlers logic/refreshing can't keep up with user inputs!