Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in ERGAS metric #2416

Closed
Andrewwango opened this issue Feb 29, 2024 · 5 comments · Fixed by #2426 or #2498
Closed

Error in ERGAS metric #2416

Andrewwango opened this issue Feb 29, 2024 · 5 comments · Fixed by #2426 or #2498
Labels
documentation Improvements or additions to documentation
Milestone

Comments

@Andrewwango
Copy link

In the definition of ERGAS, it should be \ ratio rather than * ratio

For example see ERGAS defined in Du et al, Meng et al

ergas_score = 100 * ratio * torch.sqrt(torch.sum((rmse_per_band / mean_target) ** 2, dim=1) / c)

Copy link

Hi! thanks for your contribution!, great first issue!

@SkafteNicki
Copy link
Member

Hi @Andrewwango, thanks for raising this issue.
I do not agree that there is an error in our formulation. If you look at the original paper by Wald (page 163 in the pdf) you will see that ERGAS metric is defined by multiplying the fraction and not dividing it. If you look at the paper by Du et al that you refer to, in equation (1) they ERGAS metric they multiply the ratio h/l. I am not sure why it is different in the paper by Meng et al.

I am updating the metric to reference the original paper and the specific formula we are using for calculating the metric in #2426

@SkafteNicki SkafteNicki added this to the v1.3.x milestone Mar 5, 2024
@SkafteNicki SkafteNicki added the documentation Improvements or additions to documentation label Mar 5, 2024
@Andrewwango
Copy link
Author

Andrewwango commented Mar 8, 2024

I disagree @SkafteNicki . The Wald and Du papers multiply by the ratio of pixel lengths , which is the inverse of the ratio. For example for ratio=4 the pixel length ratio h/l=1/4. The Wald paper gives examples of h/l=1/4 and h/l=1/2. Azarang et al states that h/l=1/4 for common satellites Pleiades-1A, WorldView-2, and GeoEye-1.

Therefore you should be dividing by the integer ratio. Please reopen this issue @SkafteNicki @Borda 😄

@Borda Borda reopened this Mar 8, 2024
@SkafteNicki
Copy link
Member

Hi @Andrewwango, thanks for correcting me. I am not a expert on these metrics and just saw that the original paper defined it by multiplication and division. Sorry on my part.
I make sure to fix it.

@Andrewwango
Copy link
Author

Andrewwango commented Mar 19, 2024

Hi @SkafteNicki, any progress on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
3 participants