Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added wizard section to openEO web Editor documentation #233

Merged
merged 9 commits into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 60 additions & 12 deletions Applications/WebEditor.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The openEO Web Editor can be accessed via [https://openeo.dataspace.copernicus.e

Upon initial access to the provided link, users are presented with the following screen which is further explained below in refernce to the given numbering:

![](_images/webeditor.png)
![](_images/webeditor/webeditor.png)

1. **Service Offering**

Expand Down Expand Up @@ -54,7 +54,7 @@ Upon initial access to the provided link, users are presented with the following

As previously mentioned, it is necessary to log in to interact with the server. A new window will appear when attempting to log in, as demonstrated below. While other options are sometimes available, the recommended authentication choice is the "Copernicus Data Space Ecosystem". For further information regarding various authentication methods or to seek assistance, you can always click on the "help" option at the top or contact us.

<centre><img src="_images/login.PNG" width="80%" height="60%"></centre>
<centre><img src="_images/webeditor/editor_login.PNG" width="80%" height="60%"></centre>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that this image is pretty outdated, it still contains "EGI" and "internal" options, which are dropped now

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the image



## Create a workflow
Expand All @@ -63,48 +63,96 @@ Based on their applications user can build their model by simple drag and drop m

1. Load Collection

In order to load the required collection make sure it is available by searching in the sidebar. Once you find your collection you can simply drag and drop it for carrying out further actions. In the following clip, you can observe the sequential actions are taken to accomplish the following tasks: choosing a collection of interest, defining the spatial and temporal boundaries, and filtering the necessary bands for subsequent processing. Specifically, for calculating the NDVI, the Red band (B04) and Near-Infrared (NIR) band (B08) have been selected.
a. To start a task within the openEO web editor, your first step involves ensuring you have the necessary dataset for the analysis. Thus, you must ensure the required dataset exists within the openEO database. You can do this by exploring/searching the list of available datasets from the sidebar of the interface.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use "collection" iso "dataset". Also "ensure" sounds a bit weird here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected


b. In our example, we want to calculate the Normalized Difference Vegetation Index (NDVI), so the Sentinel 2 L2A collection will be used. Once you find this collection, the next step is to drag and drop it into the Process Editor for further operations.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the difference between step a and b here, both are about finding the desired collection.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

combined and modified all the steps


![](_images/webeditor/drag_collection.png)


Now, once you load collection, there are several parameters you need to specify, including the area of interest, the temporal extent, and the selection of bands. When you click on the collection, a window pops up, where you can define the necessary parameters for subsequent processing.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you click on the collection, a window pops up,

I think this should be clicking on any load_collection parameter, or clicking the "edit" icon in top right corner

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the sentence


c. To specify the area of interest, you can choose between generating a bounding box or importing your spatial extent by dragging and dropping GeoJSON or KML files onto the map view. Here, we've provided an example involving a bounding box. There is also an option to "No filter", that will include all the data in the datacube.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No bounding box filter is generally a bad idea for novice users (the ones that use a wizard). Instead we should point out that the user should start with a sufficiently small boundingbox with respect to the resolution of the data.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the sentence


![](_images/webeditor/define_bbox.png)

d. Another parameter is Temporal extent, that imits the data to load from the collection to the specified temporal interval.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
d. Another parameter is Temporal extent, that imits the data to load from the collection to the specified temporal interval.
d. Another parameter is temporal extent to limit the data to load to the specified time window.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here as well, same as bbox: please encourage the user to start small, like a 10 or 15 day interval.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the sentence


![](_images/gif/load_collection.gif)
![](_images/webeditor/define_temporal.png)

e. In our case, for NDVI calculation, we have specifically chosen the Red band (B04) and the Near-Infrared (NIR) band (B08).

![](_images/webeditor/define_bands.png)

2. Apply Processes

Eventually, the next step involves implementing essential processes, ranging from straightforward operations like adding bands to more complex tasks such as importing or defining user-defined functions (UDFs). In the following clip, a reduce_dimension() process is employed to eliminate the temporal dimension by selecting the maximum value. The same process is utilised to reduce the band dimension after executing a series of addition, subtraction, and division operations necessary for the NDVI calculation.
Eventually, the next step involves implementing essential processes, ranging from straightforward operations like adding bands to more complex tasks such as importing or defining user-defined functions (UDFs).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Eventually, the next step involves implementing essential processes, ranging from straightforward operations like adding bands to more complex tasks such as importing or defining user-defined functions (UDFs).
The next step involves applying essential openEO processes, ranging from straightforward mathematical operations to more complex tasks such spatial or temporal aggregations.


In this task, we're using a specific process called `reduce_dimension` as a key part of the operation. This process helps us simplify our collection: first, we use it to reduce temporal dimension and second, to deal with the various bands.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In this task, we're using a specific process called `reduce_dimension` as a key part of the operation. This process helps us simplify our collection: first, we use it to reduce temporal dimension and second, to deal with the various bands.
In this task, we'll using a specific openEO process called `reduce_dimension` two times to simplify our data cube. First to reduce the temporal dimension, and second to deal with the various bands.


![](_images/webeditor/reduce_dimension.png)

In the initial process, the `reduce_dimension` algorithm eliminates the temporal dimension by selecting the maximum value using the max function.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that, scientifically speaking, it is pretty weird to first do a temporal reduction of your raw sensor values and do NDVI calculation afterwards because you will be calculating the NDVI from red and nir values that might be unrelated. I think you first should do NDVI calculation and then taking the temporal max.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the method and noted


<centre><img src="_images/webeditor/temporal_reducer.png" width="80%" height="60%"></centre>

![](_images/gif/apply_process.gif)
Following this, the same `reduce_dimension` is utilised to reduce the band dimension after executing a series of addition, subtraction, and division operations necessary for the NDVI calculation.

<centre><img src="_images/webeditor/reduce_bands.png" width="80%" height="60%"></centre>

3. Select a format

As a final step in the workflow creation, the following clip demonstrates selecting the output format. Since our application involves simple NDVI calculation, we want to save it as a GeoTiff.
As a final step in the workflow creation is to select the output format.

![](_images/webeditor/save_process.png)

Since our application involves simple NDVI calculation, we want to save it as a GeoTiff.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Since our application involves simple NDVI calculation, we want to save it as a GeoTiff.
Since our workflow eliminated the temporal dimension, we can keep things simple and just save the result as a GeoTiff.


![](_images/webeditor/save_as_gtiff.png)

![](_images/gif/save_result.gif)

## Execute the workflow

To complete the data analysis process, the final step involves executing the created workflow. This can be done in two ways: synchronously or through batch job-based method. Synchronous method allows the user to download the data directly, whereas batch job-based method enables the user to execute process as a batch. The choice of method depends on the user's preference and the size of the dataset.

![](_images/jobs.png)
![](_images/webeditor/execute_jobs.png)

In the above figure, the red box includes the two methods possible for executing the process. In this example, I used the synchronus method by directly clicking on *Run now*, which popped up a box in the bottom right corner.

Once the execution process is completed, the result is automatically saved locally. It can also be visualised in the parallel window as shown in the image below:

![](_images/result.PNG)
![](_images/webeditor/webeditor_result.PNG)


Furthermore, if you have created Batch Job, you can monitor its action from the same window.


## Data Download using Wizard

The Wizard feature at the top navigation bar allows easy creation of process graphs for common, basic use cases without having to drag and drop processes as done earlier.

:::{callout-warning}

This feature is experimental and it's not guaranteed that the generated process graphs are fully functional.

:::

At the time of this documentation release, two cases are currently available through the Wizard feature: direct downloading of data for a selected area and downloading computed spectral indices as shown in the figure below:

![](_images/webeditor/wizard.png)

You have the option to choose any one of them and enter the necessary parameters as instructed. This method makes the task considerably more straightforward than the previously mentioned workflow creation approach, as the structure is already established and you only need to input the essential parameters.

## Monitor the workflow

When a batch job is created, openEO's logs feature can help with *monitoring and debugging* of workflows. The back-end typically uses this to dump information during data processing that may be relevant for the user (e.g. warnings, resource stats, …). Like many other processes, job logs can be visualized programmatically as well as in the web editor.

As shown in the following image, you have the option to access the job logs by clicking on the icon located next to the job ID at the bottom of the interface.

![](./_images/joblist.png)
![](./_images/webeditor/joblist.png)

When clicking on it, a pane emerges on the right, as shown below:

![](./_images/joblogs.png)
![](./_images/webeditor/joblogs.png)

Moreover, users can use openEO processes like `inspect`, this will allow you to log your own information to be displayed in the job logs.
Binary file removed Applications/_images/gif/apply_process.gif
Binary file not shown.
Binary file removed Applications/_images/gif/load_collection.gif
Binary file not shown.
Binary file removed Applications/_images/gif/save_result.gif
Binary file not shown.
Binary file added Applications/_images/webeditor/define_bands.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Applications/_images/webeditor/define_bbox.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Applications/_images/webeditor/reduce_bands.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Applications/_images/webeditor/save_as_gtiff.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Applications/_images/webeditor/save_process.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Applications/_images/webeditor/wizard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.