Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event date in the question is not required, but our agents needs it #481

Open
kongzii opened this issue Oct 9, 2024 · 2 comments
Open

Comments

@kongzii
Copy link
Contributor

kongzii commented Oct 9, 2024

From the rules:

If no specific event date is given in a question, it is assumed that the resolution date will be considered the event date.

Currently, we filter out questions without event date in them here https://github.com/gnosis/prediction-market-agent-tooling/blob/main/prediction_market_agent_tooling/tools/is_predictable.py#L16.

And then we don't provide market's closing time into LLM.

So, the current implementation works fine, but it could skip some markets unnecessarily.

So if want to stick to the rules, we should:

  1. Don't require event date in the question itself.
  2. Feed market's closing date to agents.

It sounds like a straightforward and fast thing to do, but I'm worried it could have some accuracy/profitability consequences.

Because if question contains event date but we also provide closing date to the agent, it could be confused about it and predict the event according to the wrong date.

We could do some if/elsing and use the closing date only if the question doesn't contain it already, because otherwise, closing date is theoretically irrelevant.

Practically, if closing date is too much in the future, agent will have the funds locked in there.

@greenlucid
Copy link
Contributor

Note that the rules on the new default Arbitrator, 31 Jurors that was setup here, are different from the ones you referenced in your issue. So, if they're different (they're pretty similar but there might be differences?) those two need to be juggled simultaneously?

https://ipfs.kleros.io/ipfs/QmaUr6hnSVxYD899xdcn2GUVtXVjXoSXKZbce3zFtGWw4H/Question_Resolution_Policy.pdf

@kongzii
Copy link
Contributor Author

kongzii commented Oct 10, 2024

Good point, thanks! Yeah, it needs to be checked when this issue is implemented. Worst-case, we will need to accept the Arbitrator enum and make differences based on that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants