Skip to content

Evaluation Part #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ehddnr301 opened this issue Mar 14, 2025 · 1 comment
Closed

Evaluation Part #15

ehddnr301 opened this issue Mar 14, 2025 · 1 comment
Assignees
@ehddnr301
Labels
enhancement New feature or request evaluation Assessing the accuracy and efficiency of generated SQL queries.

Comments

@ehddnr301
Copy link
Collaborator

Why ❓

현재 상황 및 문제점 👿:

  • 생성되는 쿼리들이 추적되지 않음

How 🤔

해결 방안 ✅:

Image

  • 물어보새 아키텍처에서 Evaluation 부분을 구현

What 🛠️

TODO (구체화되지 않은 상태)
@ehddnr301 ehddnr301 self-assigned this Mar 14, 2025
@DShomin DShomin mentioned this issue Mar 14, 2025
Closed
7 tasks
@ehddnr301 ehddnr301 added this to the Mission: March Complete ✅ milestone Mar 15, 2025
@ehddnr301 ehddnr301 added enhancement New feature or request evaluation Assessing the accuracy and efficiency of generated SQL queries. labels Mar 16, 2025
ehddnr301 added a commit that referenced this issue Mar 16, 2025
@ehddnr301
feat: Integrate MLflow for SQL evaluation and enhance CLI functionality
a8e853d 
- Add MLflow dependency for tracking evaluation metrics
- Implement evaluation command in CLI for SQL generation model
- Create QADataset class for loading and managing evaluation datasets
- Develop Evaluator class to handle SQL evaluation and logging to MLflow
- Add documentation for MLflow setup and evaluation CLI usage
- Introduce example dataset for testing evaluation functionality

#15
@ehddnr301 ehddnr301 linked a pull request Mar 17, 2025 that will close this issue
feat: Integrate MLflow for SQL evaluation and enhance CLI functionality #23
Closed
@ehddnr301 ehddnr301 removed this from the Mission: March Complete ✅ milestone Mar 25, 2025
@ehddnr301
Copy link
Collaborator Author

우선은 필요하지 않은 기능으로 보입니다.

This was referenced Mar 26, 2025
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ehddnr301 ehddnr301

Labels
enhancement New feature or request evaluation Assessing the accuracy and efficiency of generated SQL queries.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Integrate MLflow for SQL evaluation and enhance CLI functionality CausalInferenceLab/Lang2SQL
1 participant
@ehddnr301