Fix warning about precedence of custom tasks over default ones in registry

[albertvillanova]

EDIT: See comment below: #466 (comment)

Fix warning about precedence of custom tasks over default ones in registry.

Currently, custom tasks are overwriting default ones, wheres the warning/comment says the opposite.

Fix precedence of default tasks over custom ones in registry.

Currently, custom tasks are overwriting default ones.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[albertvillanova]

I am wondering why default tasks have precedence over custom ones: I would expect that if a user creates a custom task with the same name as a default one, they would expect to overwrite the default one.

[NathanHB]

wait, not sure I follow, from the code It seems like custom tasks are overwriting the default ones right ?

[albertvillanova]

Yes, but the docs say the opposite:

Defaults tasks should overwrite custom tasks

lighteval/src/lighteval/tasks/registry.py

Lines 150 to 152 in 988fa94

	logger.warning(
	f"Following tasks ({intersection}) exists both in the default and custom tasks. Will use the default ones on conflict."
	)

[NathanHB]

You are right custom tasks should overwrite default ones I think

[albertvillanova]

I am fixing the comment/warning instead.