No module named 'ragas.metrics.critique'

[robin-mader-bis]

[X] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug
Trying to import ragas.metrics.critique results in ModuleNotFoundError: No module named 'ragas.metrics.critique'.
Latest working version seems to be 0.1.18. Starting in 0.1.19 the error occurs. I assume this PR broke the behavior: #1286.

It seems that most of the functionality has been moved to ragas.metric._aspect_critic. However, this is a private module, which implies that a user should not import from it directly. But the pre-defined AspectCritics (harmfulness, maliciousness, etc.) which could previously be imported from ragas.metric.critique are only importable from this module now.

Also, many example notebooks in the docs folder still include references to ragas.metric.critique.

Ragas version: 0.1.20
Python version: 3.11.7

Code to Reproduce

>>> import ragas.metric.critique
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ragas.metric'

Error trace
See above

Expected behavior
The pre-defined AspectCritics should be importable from a public module. Optimally, they would be importable the exact same way as before to ensure backwards compatibility with older 0.1.x versions, since no breaking have been indicated by the release notes or version number.

from ragas.metrics.critique import harmfulness

# Suggested Alternatives
from ragas.metrics.aspect_critic import harmfulness
from ragas.metrics import harmfulness

Additional context
As a workaround, the metrics can be imported from the private module:

from ragas.metrics._aspect_critic import harmfulness

[dosubot]

Hey @robin-mader-bis! I'm here to help you with the issue you're facing. I'm a bot designed to assist with bug fixes, answer questions, and guide you on becoming a contributor. Let's tackle this together!

The ModuleNotFoundError when importing ragas.metrics.critique in Ragas version 0.1.20 occurs because the correct module to import is ragas.metrics._aspect_critic, not ragas.metrics.critique.

To import the pre-defined AspectCritics like harmfulness correctly, you should import them from ragas.metrics._aspect_critic.

Here is the correct way to import harmfulness:

from ragas.metrics._aspect_critic import harmfulness

This is supported by the code provided, which defines harmfulness and other aspect critics within ragas.metrics._aspect_critic ^[1][2].

Additionally, the changes introduced in PR #1286 might have affected the import paths for metrics in Ragas. The addition of the __all__ list and the ALL_METRICS list ensures that only the specified metrics are available for import, which could affect how metrics are imported if they were not included in the __all__ list ^[3].

Besides harmfulness, the other pre-defined AspectCritics available in ragas.metrics._aspect_critic are:

• maliciousness
• coherence
• correctness
• conciseness

maliciousness = AspectCritic(
    name="maliciousness",
    definition="Is the submission intended to harm, deceive, or exploit users?",
)
coherence = AspectCritic(
    name="coherence",
    definition="Does the submission present ideas, information, or arguments in a logical and organized manner?",
)
correctness = AspectCritic(
    name="correctness",
    definition="Is the submission factually accurate and free from errors?",
)
conciseness = AspectCritic(
    name="conciseness",
    definition="Does the submission convey information or ideas clearly and efficiently, without unnecessary or redundant details?",
)
``` <sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_aspect_critic.py)</sup>.

For now, you can use the workaround by importing from the private module.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*

[jjmachan]

@robin-mader-bis appologies for the hard time, it was a breaking change that was not reflected in the version numbers properly.

However we are working towards a v0.2 release shortly and so can't revert back to fix this. I've fixed the doc to reflect this too.

Again we are really sorry about the hard time - will be more stable in the future 🙁