A few proposed modifications #20
Conversation
| @@ -29,6 +29,7 @@ def __init__( | |||
| evidence_retrieval_model: str = None, | |||
| claim_verify_model: str = None, | |||
| api_config: dict = None, | |||
| num_seed_retries: int = 3, | |||
There was a problem hiding this comment.
Allow to modify num_retries while initing FactCheck class.
| merge_evidence_text = [f"Text: {_result['snippet']} \n Source: {_result['link']}" for _result in results] | ||
| merge_evidence_text = [re.sub(r"\n+", "\n", evidence) for evidence in merge_evidence_text] | ||
| merge_evidence_text = [ | ||
| f"Text: {_result['snippet']} \n Source: {_result['link']} \n Date: {_result.get('date', 'Unknown')}" |
There was a problem hiding this comment.
Include the date of the result in the prompt. Thanks to this, OFV can verify claims such as "Did X happen before Y?"
There was a problem hiding this comment.
Hi! The inclusion of the date as a separate factor might be helpful, but I suspect it is not essential. In your example, "X happens before Y," we would expect the generated queries to be like ["When did X happen?", "When did Y happen?"]. Given such queries, retrieved evidence is expected to be sufficient to solve this problem.
Prompt for query generation:
https://github.com/Libr-AI/OpenFactVerification/blob/f620f6e5879ee1219661afc749c9fd8f658b194b/factcheck/utils/prompt/chatgpt_prompt.py#L43C1-L62
There was a problem hiding this comment.
Hey, sorry for late reply, I was on vacation.
Unfortunately, not all articles come with clear dates in them. But Serp can return their creation date anyway.
There was a problem hiding this comment.
Maybe this could be available behind a flag in FactCheck's initialisation?
| @@ -1,7 +1,9 @@ | |||
| [tool.poetry] | |||
| package-mode = false | |||
There was a problem hiding this comment.
Allow the installation of OFV as a package in another repository.
| @@ -24,7 +26,7 @@ python = "^3.9" | |||
| anthropic = "^0.23.1" | |||
| backoff = "^2.2.1" | |||
| bs4 = "^0.0.2" | |||
| Flask = "^3.0.3" | |||
| Flask = { version = "^3.0.3", optional = true } | |||
There was a problem hiding this comment.
Make Flask optional to reduce conflicts with other repositories, as Flask is not required for the core OFV functionality.
|
Thank you for your contribution! |
Hi, I'm working on a bot that should resolve prediction markets after some event has happened. And your project seems like a great fit for such a use case.
I did a few experiments, and they look very promising.
This PR contains a few modifications; I'll comment on them in the code for clarity.
If you would be so kind to provide feedback about the proposed changes, it would be great 🤞 Of course, I'm happy to add any modifications/updates that you may comment on before merging.