Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

median budget cross goes astray with many repetitions #2287

Closed
nikohansen opened this issue May 15, 2024 · 1 comment
Closed

median budget cross goes astray with many repetitions #2287

nikohansen opened this issue May 15, 2024 · 1 comment

Comments

@nikohansen
Copy link
Contributor

nikohansen commented May 15, 2024

See #2258 for considerations what to display. Otherwise, taking the median of the median of the median... is always less prone to be dominated by many repetitions of a single instance (or function). It would not work for f7 itself which is probably a good thing?

When making instance repetitions, see e.g. #2208, until a budget is exhausted, some algorithms will make very many repetitions on the step-ellipsoid, because they immediately stop on a plateau. As a consequence, the median budget over all trials from different problems becomes very low and is somewhat quite misleading as it gives more weight to the functions with a large number of trial repetitions.

A possible fix would be to take the median of the median of the problem instance budgets instead of the median of all problem instances.

@nikohansen
Copy link
Contributor Author

nikohansen commented Jun 27, 2024

Commit 3882ef6 puts the crosses to the median of max(median(RT_u), median(RT_s)) where median(RT_*) is the runtime median over all (15+) instances of a DataSet, u=unsuccessful and s=successful subset.

nikohansen added a commit that referenced this issue Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant