Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implementation of evaluation metrics for Knowledge Graphs #41

Open
4 tasks
debrupf2946 opened this issue Sep 23, 2024 · 2 comments
Open
4 tasks

Comments

@debrupf2946
Copy link
Collaborator

Implement Python Code for Knowledge Graph Evaluation Metric

Description

This issue invites contributors to implement a Python version of evaluation metrics proposed by other contributors. These metrics are essential for assessing the quality and performance of our knowledge graphs, and your implementation will directly contribute to enhancing our project's evaluation framework.If you would like to propose metrics, check #40

Objective

  • Select a metric proposal submitted by another contributor (check the open issues with metric-proposal label).
  • Develop a Python implementation of the selected metric.
  • Ensure the implementation is efficient, well-documented, and tested against sample data.

Guidelines

  1. Select a Metric: Review the issues tagged with metric-proposal to choose an evaluation metric that interests you and has not been implemented yet.
  2. Understand the Metric: Carefully read the detailed review provided in the metric proposal issue, focusing on what it measures, how it measures, and any implementation notes.
  3. Develop the Code: Implement the metric in Python, ensuring it aligns with the described methodology and requirements.
  4. Document Your Work:
    • Include docstrings to explain your code clearly.
    • Provide comments where necessary to make the code understandable.
    • Jupyter notebook implementation of metric.
  5. Submit a Pull Request (PR):
    • Reference the original metric proposal issue in your PR.
    • Include a brief description of your implementation, any challenges faced, and how you tested the metric.

Resources

  • Review the metric proposal issue to understand the metric details.
  • Use the provided research papers and any additional resources needed for accurate implementation.

Submission Checklist

  • Select and claim a metric from the metric-proposal issues.
  • Implement the metric in Python.
  • Add comprehensive documentation
  • Submit a PR referencing the metric proposal issue.

We appreciate your contributions and look forward to your implementations enhancing our knowledge graph evaluation capabilities!

@Kitsunnneee
Copy link

Hello @debrupf2946, Will it be okay to assign this issue to me, sir? I have previous experience in research in Hybrid-RAG systems and LLM fine tuning.

@debrupf2946
Copy link
Collaborator Author

@kmehant Can I assign @Kitsunnneee for this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants